mlsanityData to 14 Oct 2026

Explaining Explanations: An Overview of Interpretability of Machine Learning

arXiv 1806.00069 ↗First posted 31 May 2018cs.AI cs.LG stat.MLcited by 508

Not on the map. Why?

Abstract (arXiv)

There has recently been a surge of work in explanatory artificial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, providing explanations of their decisions in some level of detail. These explanations are important to ensure algorithmic fairness, identify potential bias/problems in the training data, and to ensure that the algorithms perform as expected. However, explanations produced by these systems is neither standardized nor systematically assessed. In an effort to create best practices and identify open challenges, we provide our definition of explainability and show how it can be used to classify existing literature. We discuss why current approaches to explanatory methods especially for deep neural networks are insufficient. Finally, based on our survey, we conclude with suggested future research directions for explanatory artificial intelligence.

Key facts. Cited by 508 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 111; 2022: 106; 2023: 94; 2024: 89; 2025: 73; 2026: 35.

508 papers.

Sort:

By area:

  1. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  2. 2022Visual Classification via Description from Large Language Modelscited by 336
  3. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  4. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  5. 2021Unsolved Problems in ML Safetycited by 292
  6. 2023Faithful Chain-of-Thought Reasoningcited by 288
  7. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 159
  8. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  9. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  10. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  11. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  12. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
38 more
  1. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  2. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 118
  3. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  4. 2021Trustworthy AI: A Computational Perspectivecited by 93
  5. 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 91
  6. 2021Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classificationcited by 89
  7. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 85
  8. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  9. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  10. 2021A Survey on Interpretable Reinforcement Learningcited by 73
  11. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
  12. 2022Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderationcited by 67
  13. 2024Attention Heads of Large Language Models: A Surveycited by 67
  14. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 66
  15. 2021This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networkscited by 65
  16. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  17. 2024Large Language Model Agent for Hyper-Parameter Optimizationcited by 64
  18. 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
  19. 2021Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applicationscited by 59
  20. 2021Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Makingcited by 58
  21. 2023On Early Detection of Hallucinations in Factual Question Answeringcited by 56
  22. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  23. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  24. 2021Exploiting Explanations for Model Inversion Attackscited by 49
  25. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  26. 2021Discretized Integrated Gradients for Explaining Language Modelscited by 46
  27. 2022Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capabilitycited by 46
  28. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  29. 2021Logic Explained Networkscited by 45
  30. 2021CoNIC: Colon Nuclei Identification and Counting Challenge 2022cited by 45
  31. 2024TokenSHAP: Interpreting Large Language Models with Monte Carlo Shapley Value Estimationcited by 42
  32. 2021Evaluating Robustness of Counterfactual Explanationscited by 41
  33. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  34. 2021Explainable Artificial Intelligence Approaches: A Surveycited by 40
  35. 2021STEEX: Steering Counterfactual Explanations with Semanticscited by 40
  36. 2022Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanationscited by 40
  37. 2022Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AIcited by 40
  38. 2021A Survey on the Robustness of Feature Importance and Counterfactual Explanationscited by 39

50 most cited of 508 All 508 (JSON)