mlsanityData to 14 Oct 2026

Problems with Shapley-value-based explanations as feature importance measures

arXiv 2002.11097 ↗First posted 25 Feb 2020cs.AI cs.LG stat.MLcited by 200

Not on the map. Why?

Abstract (arXiv)

Game-theoretic formulations of feature importance have become popular as a way to "explain" machine learning models. These methods define a cooperative game between the features of a model and distribute influence among these input elements using some form of the game's unique Shapley values. Justification for these methods rests on two pillars: their desirable mathematical properties, and their applicability to specific motivations for explanations. We show that mathematical problems arise when Shapley values are used for feature importance and that the solutions to mitigate these necessarily induce further complexity, such as the need for causal reasoning. We also draw on additional literature to argue that Shapley values do not provide explanations which suit human-centric goals of explainability.

Key facts. Cited by 200 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 44; 2022: 39; 2023: 36; 2024: 36; 2025: 26; 2026: 19.

200 papers.

Sort:

By area:

  1. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  2. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  3. 2022The Shapley Value in Machine Learningcited by 98
  4. 2022Impossibility Theorems for Feature Attributioncited by 78
  5. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  6. 2022From Shapley Values to Generalized Additive Models and backcited by 60
  7. 2021Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactualscited by 55
  8. 2021Shapley values for feature selection: The good, the bad, and the axiomscited by 54
  9. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  10. 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 53
  11. 2021Local Explanations via Necessity and Sufficiency: Unifying Theory and Practicecited by 45
  12. 2024shapiq: Shapley Interactions for Machine Learningcited by 43
38 more
  1. 2021Shapley Counterfactual Credits for Multi-Agent Reinforcement Learningcited by 38
  2. 2023Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Valuecited by 38
  3. 2021Explaining a Series of Models by Propagating Shapley Valuescited by 37
  4. 2022WeightedSHAP: analyzing and improving Shapley based feature attributionscited by 36
  5. 2021On the Complexity of SHAP-Score-Based Explanations: Tractability via Knowledge Compilation and Non-Approximability Resultscited by 32
  6. 2023The Inadequacy of Shapley Values for Explainabilitycited by 31
  7. 2024Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensemblescited by 31
  8. 2021Fast TreeSHAP: Accelerating SHAP Value Computation for Treescited by 29
  9. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  10. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
  11. 2021Designing Inherently Interpretable Machine Learning Modelscited by 23
  12. 2023Feature Importance: A Closer Look at Shapley Values and LOCOcited by 23
  13. 2023Explainability is NOT a Gamecited by 23
  14. 2021Counterfactual Shapley Additive Explanationscited by 22
  15. 2024KernelSHAP-IQ: Weighted Least-Square Optimization for Shapley Interactionscited by 22
  16. 2023Tell Me a Story! Narrative-Driven XAI with Large Language Modelscited by 20
  17. 2022Use-Case-Grounded Simulations for Explanation Evaluationcited by 19
  18. 2023Explaining the Uncertain: Stochastic Shapley Values for Gaussian Process Modelscited by 19
  19. 2021Rational Shapley Valuescited by 18
  20. 2022Unifying local and global model explanations by functional decomposition of low dimensional structurescited by 18
  21. 2021PredDiff: Explanations and Interactions from Conditional Expectationscited by 17
  22. 2021On the Lack of Robust Interpretability of Neural Text Classifierscited by 17
  23. 2023Explaining Quantum Circuits with Shapley Values: Towards Explainable Quantum Machine Learningcited by 17
  24. 2022Explainability in Music Recommender Systemscited by 15
  25. 2023A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Themcited by 15
  26. 2022Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-makingcited by 14
  27. 2022Deletion and Insertion Tests in Regression Modelscited by 14
  28. 2021Cohort Shapley value for algorithmic fairnesscited by 13
  29. 2023Setting the Right Expectations: Algorithmic Recourse Over Timecited by 13
  30. 2021FairCanary: Rapid Continuous Explainable Fairnesscited by 12
  31. 2022Statistical Aspects of SHAP: Functional ANOVA for Model Interpretationcited by 12
  32. 2021Towards Rigorous Interpretations: a Formalisation of Feature Attributioncited by 11
  33. 2021Evaluating the Values of Sources in Transfer Learningcited by 11
  34. 2022Interpretable models for extrapolation in scientific machine learningcited by 11
  35. 2024How should AI decisions be explained? Requirements for Explanations from the Perspective of European Lawcited by 11
  36. 2024Understanding Individual Agent Importance in Multi-Agent System via Counterfactual Reasoningcited by 11
  37. 2021Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?cited by 10
  38. 2021Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decompositioncited by 9

50 most cited of 200 All 200 (JSON)