mlsanityData to 7 Oct 2026

Feature relevance quantification in explainable AI: A causal problem

arXiv 1910.13413 ↗First posted 29 Oct 2019stat.ML cs.LGcited by 164

Not on the map. Why?

Abstract (arXiv)

We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditional probabilities and try a clarification based on Pearl's seminal work on causality. We conclude that unconditional rather than conditional expectations provide the right notion of dropping features in contradiction to the theoretical justification of the software package SHAP. Parts of SHAP are unaffected because unconditional expectations (which we argue to be conceptually right) are used as approximation for the conditional ones, which encouraged others to `improve' SHAP in a way that we believe to be flawed.

Key facts. Cited by 164 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 33; 2022: 23; 2023: 32; 2024: 20; 2025: 20; 2026: 36.

164 papers.

Sort:

By area:

  1. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  2. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 123
  3. 2022Algorithms to estimate Shapley value feature attributionscited by 114
  4. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  5. 2022Impossibility Theorems for Feature Attributioncited by 78
  6. 2021Synthetic Benchmarks for Scientific Research in Explainable Machine Learningcited by 59
  7. 2022From Shapley Values to Generalized Additive Models and backcited by 59
  8. 2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
  9. 2022DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal modelscited by 40
  10. 2021Why did the distribution change?cited by 37
  11. 2021Explaining a Series of Models by Propagating Shapley Valuescited by 37
  12. 2022WeightedSHAP: analyzing and improving Shapley based feature attributionscited by 36
38 more
  1. 2023Explaining Predictive Uncertainty with Information Theoretic Shapley Valuescited by 36
  2. 2021RKHS-SHAP: Shapley Values for Kernel Methodscited by 30
  3. 2021Fast TreeSHAP: Accelerating SHAP Value Computation for Treescited by 28
  4. 2021On Locality of Local Explanation Modelscited by 25
  5. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 25
  6. 2021You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Predictioncited by 23
  7. 2022Marrying Fairness and Explainability in Supervised Learningcited by 23
  8. 2024Provably Accurate Shapley Value Estimation via Leverage Score Samplingcited by 23
  9. 2021SHAFF: Fast and consistent SHApley eFfect estimates via random Forestscited by 22
  10. 2021Counterfactual Shapley Additive Explanationscited by 22
  11. 2021groupShapley: Efficient prediction explanation with Shapley values for feature groupscited by 21
  12. 2022Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomenacited by 21
  13. 2024Optimal ablation for interpretabilitycited by 21
  14. 2021Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloudcited by 19
  15. 2025Regression-adjusted Monte Carlo Estimators for Shapley Values and Probabilistic Valuescited by 19
  16. 2021Rational Shapley Valuescited by 18
  17. 2022Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detectioncited by 18
  18. 2022Unifying local and global model explanations by functional decomposition of low dimensional structurescited by 18
  19. 2022"Why did the Model Fail?": Attributing Model Performance Changes to Distribution Shiftscited by 18
  20. 2023The role of causality in explainable artificial intelligencecited by 17
  21. 2021PredDiff: Explanations and Interactions from Conditional Expectationscited by 16
  22. 2021Fooling Partial Dependence via Data Poisoningcited by 16
  23. 2021Fast Hierarchical Games for Image Explanationscited by 15
  24. 2021Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution Viewcited by 15
  25. 2023On Learning Necessary and Sufficient Causal Graphscited by 14
  26. 2023A Review of the Role of Causality in Developing Trustworthy AI Systemscited by 14
  27. 2023Theoretical Behavior of XAI Methods in the Presence of Suppressor Variablescited by 12
  28. 2023DRCFS: Doubly Robust Causal Feature Selectioncited by 12
  29. 2024Learning to Understand: Identifying Interactions via the M"obius Transformcited by 12
  30. 2025Exact Computation of Any-Order Shapley Interactions for Graph Neural Networkscited by 12
  31. 2024Hard to Explain: On the Computational Hardness of In-Distribution Model Interpretationcited by 11
  32. 2024Unifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theorycited by 10
  33. 2021Accurate Shapley Values for explaining tree-based modelscited by 9
  34. 2021Cause and Effect: Hierarchical Concept-based Explanation of Neural Networkscited by 8
  35. 2022Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streamscited by 8
  36. 2023Manifold Restricted Interventional Shapley Valuescited by 8
  37. 2023Feature Interactions Reveal Linguistic Structure in Language Modelscited by 8
  38. 2024SHAP values via sparse Fourier representationcited by 8

50 most cited of 164 All 164 (JSON)