mlsanityData to 7 Oct 2026

Towards A Rigorous Science of Interpretable Machine Learning

arXiv 1702.08608 ↗First posted 28 Feb 2017stat.ML cs.AI cs.LGcited by 1,400

Not on the map. Why?

Cited by 39 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.

Key facts. Cited by 1,400 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 3; 2019: 1; 2020: 0; 2021: 230; 2022: 229; 2023: 217; 2024: 234; 2025: 279; 2026: 207.

1,400 papers.

Sort:

By area:

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  2. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  3. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  4. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  5. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  6. 2023Explainability for Large Language Models: A Surveycited by 331
  7. 2021Trustworthy AI: From Principles to Practicescited by 206
  8. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 177
  9. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 156
  10. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  11. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  12. 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
38 more
  1. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
  2. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  3. 2021CrypTen: Secure Multi-Party Computation Meets Machine Learningcited by 131
  4. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  5. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  6. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  7. 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
  8. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  9. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  10. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  11. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 123
  12. 2021Acquisition of Chess Knowledge in AlphaZerocited by 120
  13. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  14. 2021CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networkscited by 118
  15. 2024Wav-KAN: Wavelet Kolmogorov-Arnold Networkscited by 118
  16. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  17. 2022Algorithms to estimate Shapley value feature attributionscited by 114
  18. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  19. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  20. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 107
  21. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  22. 2024Are self-explanations from Large Language Models faithful?cited by 101
  23. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  24. 2024Universal Neurons in GPT2 Language Modelscited by 100
  25. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  26. 2021Trustworthy AI: A Computational Perspectivecited by 93
  27. 2022Interpreting Language Models with Contrastive Explanationscited by 91
  28. 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 90
  29. 2024NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Securitycited by 90
  30. 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 87
  31. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 85
  32. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  33. 2023Machine Unlearning: Solutions and Challengescited by 85
  34. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  35. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  36. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  37. 2021If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniquescited by 82
  38. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82

50 most cited of 1,400 All 1,400 (JSON)