-
Deep-learning models based on artificial neural networks have grown massively, but their complexity means it is unclear if they are working correctly.
-
Researchers have developed 3 ‘explanation methods’ to try and show how and whether machine learning is working – feature attribution, counterfactual explanation and sample importance explanations.
-
This is helping decision makers know when to trust AI and put its guidance in practice.
-
As machine learning enters into more disciplines – from healthcare to education – explanation methods are becoming more important, but improvements are still needed.
Copyright: weforum.org – “How to tell if artificial intelligence is working the way we want it to”
About a decade ago, deep-learning models started achieving superhuman results on all sorts of tasks, from beating world-champion board game players to outperforming doctors at diagnosing breast cancer.
These powerful deep-learning models are usually based on artificial neural networks, which were first proposed in the 1940s and have become a popular type of machine learning. A computer learns to process data using layers of interconnected nodes, or neurons, that mimic the human brain.
As the field of machine learning has grown, artificial neural networks have grown along with it.
Deep-learning models are now often composed of millions or billions of interconnected nodes in many layers that are trained to perform detection or classification tasks using vast amounts of data. But because the models are so enormously complex, even the researchers who design them don’t fully understand how they work. This makes it hard to know whether they are working correctly.
For instance, maybe a model designed to help physicians diagnose patients correctly predicted that a skin lesion was cancerous, but it did so by focusing on an unrelated mark that happens to frequently occur when there is cancerous tissue in a photo, rather than on the cancerous tissue itself. This is known as a spurious correlation. The model gets the prediction right, but it does so for the wrong reason. In a real clinical setting where the mark does not appear on cancer-positive images, it could result in missed diagnoses.
Thank you for reading this post, don't forget to subscribe to our AI NAVIGATOR!
With so much uncertainty swirling around these so-called “black-box” models, how can one unravel what’s going on inside the box.
This puzzle has led to a new and rapidly growing area of study in which researchers develop and test explanation methods (also called interpretability methods) that seek to shed some light on how black-box machine-learning models make predictions.
What are explanation methods?
At their most basic level, explanation methods are either global or local. A local explanation method focuses on explaining how the model made one specific prediction, while global explanations seek to describe the overall behaviour of an entire model. This is often done by developing a separate, simpler (and hopefully understandable) model that mimics the larger, black-box model.
But because deep learning models work in fundamentally complex and nonlinear ways, developing an effective global explanation model is particularly challenging. This has led researchers to turn much of their recent focus onto local explanation methods instead, explains Yilun Zhou, a graduate student in the Interactive Robotics Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL) who studies models, algorithms, and evaluations in interpretable machine learning.[…]
Read more: www.weforum.org
Deep-learning models based on artificial neural networks have grown massively, but their complexity means it is unclear if they are working correctly.
Researchers have developed 3 ‘explanation methods’ to try and show how and whether machine learning is working – feature attribution, counterfactual explanation and sample importance explanations.
This is helping decision makers know when to trust AI and put its guidance in practice.
As machine learning enters into more disciplines – from healthcare to education – explanation methods are becoming more important, but improvements are still needed.
Copyright: weforum.org – “How to tell if artificial intelligence is working the way we want it to”
These powerful deep-learning models are usually based on artificial neural networks, which were first proposed in the 1940s and have become a popular type of machine learning. A computer learns to process data using layers of interconnected nodes, or neurons, that mimic the human brain.
As the field of machine learning has grown, artificial neural networks have grown along with it.
Deep-learning models are now often composed of millions or billions of interconnected nodes in many layers that are trained to perform detection or classification tasks using vast amounts of data. But because the models are so enormously complex, even the researchers who design them don’t fully understand how they work. This makes it hard to know whether they are working correctly.
For instance, maybe a model designed to help physicians diagnose patients correctly predicted that a skin lesion was cancerous, but it did so by focusing on an unrelated mark that happens to frequently occur when there is cancerous tissue in a photo, rather than on the cancerous tissue itself. This is known as a spurious correlation. The model gets the prediction right, but it does so for the wrong reason. In a real clinical setting where the mark does not appear on cancer-positive images, it could result in missed diagnoses.
Thank you for reading this post, don't forget to subscribe to our AI NAVIGATOR!
With so much uncertainty swirling around these so-called “black-box” models, how can one unravel what’s going on inside the box.
This puzzle has led to a new and rapidly growing area of study in which researchers develop and test explanation methods (also called interpretability methods) that seek to shed some light on how black-box machine-learning models make predictions.
What are explanation methods?
At their most basic level, explanation methods are either global or local. A local explanation method focuses on explaining how the model made one specific prediction, while global explanations seek to describe the overall behaviour of an entire model. This is often done by developing a separate, simpler (and hopefully understandable) model that mimics the larger, black-box model.
But because deep learning models work in fundamentally complex and nonlinear ways, developing an effective global explanation model is particularly challenging. This has led researchers to turn much of their recent focus onto local explanation methods instead, explains Yilun Zhou, a graduate student in the Interactive Robotics Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL) who studies models, algorithms, and evaluations in interpretable machine learning.[…]
Read more: www.weforum.org
Share this: