# Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges

arXiv: https://arxiv.org/abs/2010.09337 · HTML: https://mlsanity.com/paper/2010.09337/

## Key facts

- Cited by 120 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 30 of 120)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [Evaluation and Improvement of Interpretability for Self-Explainable Part-Prototype Networks](https://mlsanity.com/paper/2212.05946/) (2022)
- [Trust in AI: Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2403.14680/) (2024)
- [A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges](https://mlsanity.com/paper/2210.12089/) (2022)
- [Choice modelling in the age of machine learning -- discussion paper](https://mlsanity.com/paper/2101.11948/) (2021)
- [FuXi-Extreme: Improving extreme rainfall and wind forecasts with diffusion model](https://mlsanity.com/paper/2310.19822/) (2023)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Neurosymbolic AI for Reasoning over Knowledge Graphs: A Survey](https://mlsanity.com/paper/2302.07200/) (2023)
- [XAI for All: Can Large Language Models Simplify Explainable AI?](https://mlsanity.com/paper/2401.13110/) (2024)
- [A Survey on Aspect-Based Sentiment Classification](https://mlsanity.com/paper/2203.14266/) (2022)
- [BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks](https://mlsanity.com/paper/2103.07224/) (2021)
- [CounterNet: End-to-End Training of Prediction Aware Counterfactual Explanations](https://mlsanity.com/paper/2109.07557/) (2021)
- [The role of causality in explainable artificial intelligence](https://mlsanity.com/paper/2309.09901/) (2023)
- [Uncertainty Estimation and Out-of-Distribution Detection for Counterfactual Explanations: Pitfalls and Solutions](https://mlsanity.com/paper/2107.09734/) (2021)
- [Comparing Explanation Methods for Traditional Machine Learning Models Part 1: An Overview of Current Methods and Quantifying Their Disagreement](https://mlsanity.com/paper/2211.08943/) (2022)
- [Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Dataset](https://mlsanity.com/paper/2204.11642/) (2022)
- [On the Interpretability of Quantum Neural Networks](https://mlsanity.com/paper/2308.11098/) (2023)
- [Toward a Theory of Causation for Interpreting Neural Code Models](https://mlsanity.com/paper/2302.03788/) (2023)
- [Interpretability of Machine Learning: Recent Advances and Future Prospects](https://mlsanity.com/paper/2305.00537/) (2023)
- [Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaboration](https://mlsanity.com/paper/2403.04629/) (2024)
- [A Prescriptive Learning Analytics Framework: Beyond Predictive Modelling and onto Explainable AI with Prescriptive Analytics and ChatGPT](https://mlsanity.com/paper/2208.14582/) (2022)
- [DALE: Differential Accumulated Local Effects for efficient and accurate global explanations](https://mlsanity.com/paper/2210.04542/) (2022)
- [Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations](https://mlsanity.com/paper/2407.08983/) (2024)
- [Evaluating and Explaining Large Language Models for Code Using Syntactic Structures](https://mlsanity.com/paper/2308.03873/) (2023)
- [RHALE: Robust and Heterogeneity-aware Accumulated Local Effects](https://mlsanity.com/paper/2309.11193/) (2023)
- [Shapley variable importance clouds for interpretable machine learning](https://mlsanity.com/paper/2110.02484/) (2021)
- [Interpretability with full complexity by constraining feature information](https://mlsanity.com/paper/2211.17264/) (2022)
- [Multi-modal Machine Learning for Vehicle Rating Predictions Using Image, Text, and Parametric Data](https://mlsanity.com/paper/2305.15218/) (2023)

## Abstract (arXiv)

We present a brief history of the field of interpretable machine learning (IML), give an overview of state-of-the-art interpretation methods, and discuss challenges. Research in IML has boomed in recent years. As young as the field is, it has over 200 years old roots in regression modeling and rule-based machine learning, starting in the 1960s. Recently, many new IML methods have been proposed, many of them model-agnostic, but also interpretation techniques specific to deep learning and tree-based ensembles. IML methods either directly analyze model components, study sensitivity to input perturbations, or analyze local or global surrogate approximations of the ML model. The field approaches a state of readiness and stability, with many methods not only proposed in research, but also implemented in open-source software. But many important challenges remain for IML, such as dealing with dependent features, causal interpretation, and uncertainty estimation, which need to be resolved for its successful application to scientific problems. A further challenge is a missing rigorous definition of interpretability, which is accepted by the community. To address the challenges and advance the field, we urge to recall our roots of interpretable, data-driven modeling in statistics and (rule-based) ML, but also to consider other areas such as sensitivity analysis, causal inference, and the social sciences.