# Individual Explanations in Machine Learning Models: A Survey for Practitioners

arXiv: https://arxiv.org/abs/2104.04144 · HTML: https://mlsanity.com/paper/2104.04144/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 22 of 22)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations](https://mlsanity.com/paper/1703.03717/) (2017)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [Counterfactual Fairness in Text Classification through Robustness](https://mlsanity.com/paper/1809.10610/) (2018)
- [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019)
- [Visualizing Deep Networks by Optimizing with Integrated Gradients](https://mlsanity.com/paper/1905.00954/) (2019)
- [Visualizing the Feature Importance for Black Box Models](https://mlsanity.com/paper/1804.06620/) (2018)
- [Learning what and where to attend](https://mlsanity.com/paper/1805.08819/) (2018)

## Built on by (most cited first, 6 of 6)

- [AI Readiness in Healthcare through Storytelling XAI](https://mlsanity.com/paper/2410.18725/) (2024)
- [Learning Channel Importance for High Content Imaging with Interpretable Deep Input Channel Mixing](https://mlsanity.com/paper/2308.16637/) (2023)
- [From Explanation to Action: An End-to-End Human-in-the-loop Framework for Anomaly Reasoning and Management](https://mlsanity.com/paper/2304.03368/) (2023)
- [An Introduction to Natural Language Processing Techniques and Framework for Clinical Implementation in Radiation Oncology](https://mlsanity.com/paper/2311.02205/) (2023)
- [Model Explanations via the Axiomatic Causal Lens](https://mlsanity.com/paper/2109.03890/) (2021)
- [Evaluating Explanatory Capabilities of Machine Learning Models in Medical Diagnostics: A Human-in-the-Loop Approach](https://mlsanity.com/paper/2403.19820/) (2024)

## Abstract (arXiv)

In recent years, the use of sophisticated statistical models that influence decisions in domains of high societal relevance is on the rise. Although these models can often bring substantial improvements in the accuracy and efficiency of organizations, many governments, institutions, and companies are reluctant to their adoption as their output is often difficult to explain in human-interpretable ways. Hence, these models are often regarded as black-boxes, in the sense that their internal mechanisms can be opaque to human audit. In real-world applications, particularly in domains where decisions can have a sensitive impact--e.g., criminal justice, estimating credit scores, insurance risk, health risks, etc.--model interpretability is desired. Recently, the academic literature has proposed a substantial amount of methods for providing interpretable explanations to machine learning models. This survey reviews the most relevant and novel methods that form the state-of-the-art for addressing the particular problem of explaining individual instances in machine learning. It seeks to provide a succinct review that can guide data science and machine learning practitioners in the search for appropriate methods to their problem domain.