# Interpretability and Explainability: A Machine Learning Zoo Mini-tour

arXiv: https://arxiv.org/abs/2012.01805 · HTML: https://mlsanity.com/paper/2012.01805/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 30 of 55)

- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction](https://mlsanity.com/paper/2409.00265/) (2024)
- [Logic Explained Networks](https://mlsanity.com/paper/2108.05149/) (2021)
- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [Sparse Visual Counterfactual Explanations in Image Space](https://mlsanity.com/paper/2205.07972/) (2022)
- [The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models](https://mlsanity.com/paper/2404.03189/) (2024)
- [What Makes a Good Explanation?: A Harmonized View of Properties of Explanations](https://mlsanity.com/paper/2211.05667/) (2022)
- [Neurosymbolic AI for Reasoning over Knowledge Graphs: A Survey](https://mlsanity.com/paper/2302.07200/) (2023)
- [Explaining Quantum Circuits with Shapley Values: Towards Explainable Quantum Machine Learning](https://mlsanity.com/paper/2301.09138/) (2023)
- [How to Find a Good Explanation for Clustering?](https://mlsanity.com/paper/2112.06580/) (2021)
- [FUTURE-AI: Guiding Principles and Consensus Recommendations for Trustworthy Artificial Intelligence in Medical Imaging](https://mlsanity.com/paper/2109.09658/) (2021)
- [Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligence](https://mlsanity.com/paper/2112.14466/) (2021)
- [System Cards for AI-Based Decision-Making for Public Policy](https://mlsanity.com/paper/2203.04754/) (2022)
- [Ensembles of Random SHAPs](https://mlsanity.com/paper/2103.03302/) (2021)
- [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions](https://mlsanity.com/paper/2107.09546/) (2021)
- [Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance](https://mlsanity.com/paper/2207.01482/) (2022)
- [Survey on Leveraging Uncertainty Estimation Towards Trustworthy Deep Neural Networks: The Case of Reject Option and Post-training Processing](https://mlsanity.com/paper/2304.04906/) (2023)
- [An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data](https://mlsanity.com/paper/2106.09111/) (2021)
- [The Contextual Lasso: Sparse Linear Models via Deep Neural Networks](https://mlsanity.com/paper/2302.00878/) (2023)
- [A systematic review of biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data](https://mlsanity.com/paper/2207.00812/) (2022)
- [Symbolic Regression as Feature Engineering Method for Machine and Deep Learning Regression Tasks](https://mlsanity.com/paper/2311.06028/) (2023)
- [TorchEsegeta: Framework for Interpretability and Explainability of Image-based Deep Learning Models](https://mlsanity.com/paper/2110.08429/) (2021)
- [Can Requirements Engineering Support Explainable Artificial Intelligence? Towards a User-Centric Approach for Explainability Requirements](https://mlsanity.com/paper/2206.01507/) (2022)
- [Neurosymbolic AI for Reasoning on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2307.08411/) (2023)
- [R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut](https://mlsanity.com/paper/2307.09050/) (2023)
- [Interpretability in Symbolic Regression: a benchmark of Explanatory Methods using the Feynman data set](https://mlsanity.com/paper/2404.05908/) (2024)
- [From Logits to Hierarchies: Hierarchical Clustering made Simple](https://mlsanity.com/paper/2410.07858/) (2024)
- [Can sparse autoencoders make sense of gene expression latent variable models?](https://mlsanity.com/paper/2410.11468/) (2024)
- [MLHOps: Machine Learning for Healthcare Operations](https://mlsanity.com/paper/2305.02474/) (2023)
- [torchosr -- a PyTorch extension package for Open Set Recognition models evaluation in Python](https://mlsanity.com/paper/2305.09646/) (2023)

## Abstract (arXiv)

In this review, we examine the problem of designing interpretable and explainable machine learning models. Interpretability and explainability lie at the core of many machine learning and statistical applications in medicine, economics, law, and natural sciences. Although interpretability and explainability have escaped a clear universal definition, many techniques motivated by these properties have been developed over the recent 30 years with the focus currently shifting towards deep learning methods. In this review, we emphasise the divide between interpretability and explainability and illustrate these two different research directions with concrete examples of the state-of-the-art. The review is intended for a general machine learning audience with interest in exploring the problems of interpretation and explanation beyond logistic regression or random forest variable importance. This work is not an exhaustive literature survey, but rather a primer focusing selectively on certain lines of research which the authors found interesting or informative.