mlsanityData to 14 Oct 2026

A Unified Approach to Interpreting Model Predictions

arXiv 1705.07874 ↗First posted 22 May 2017cs.AI cs.LG stat.MLcited by 6,555

Not on the map. Why?

Cited by 77 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Understanding why a model makes a certain prediction can be as crucial as the prediction's accuracy in many applications. However, the highest accuracy for large modern datasets is often achieved by complex models that even experts struggle to interpret, such as ensemble or deep learning models, creating a tension between accuracy and interpretability. In response, various methods have recently been proposed to help users interpret the predictions of complex models, but it is often unclear how these methods are related and when one method is preferable over another. To address this problem, we present a unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations). SHAP assigns each feature an importance value for a particular prediction. Its novel components include: (1) the identification of a new class of additive feature importance measures, and (2) theoretical results showing there is a unique solution in this class with a set of desirable properties. The new class unifies six existing methods, notable because several recent methods in the class lack the proposed desirable properties. Based on insights from this unification, we present new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.

Key facts. Cited by 6,555 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 694; 2022: 827; 2023: 1052; 2024: 1190; 2025: 1402; 2026: 1389.

6,555 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  2. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  3. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  4. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  5. 2021Causal Abstractions of Neural Networkscited by 422
  6. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
  7. 2023Explainability for Large Language Models: A Surveycited by 333
  8. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 320
  9. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  10. 2023Label-Free Concept Bottleneck Modelscited by 285
  11. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  12. 2021On Explainability of Graph Neural Networks via Subgraph Explorationscited by 281
38 more
  1. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 267
  2. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 232
  3. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  4. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 222
  5. 2021Trustworthy AI: From Principles to Practicescited by 207
  6. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  7. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
  8. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  9. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  10. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
  11. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 178
  12. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  13. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  14. 2021Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learningcited by 165
  15. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 160
  16. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 159
  17. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  18. 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
  19. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  20. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  21. 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
  22. 2022Data Banzhaf: A Robust Data Valuation Framework for Machine Learningcited by 132
  23. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  24. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 128
  25. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 126
  26. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 125
  27. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  28. 2021Generative Causal Explanations for Graph Neural Networkscited by 124
  29. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  30. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  31. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 123
  32. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  33. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  34. 2021CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networkscited by 119
  35. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  36. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 118
  37. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  38. 2022Algorithms to estimate Shapley value feature attributionscited by 115

50 most cited of 6,555 All 6,555 (JSON)