# Feature relevance quantification in explainable AI: A causal problem

arXiv: https://arxiv.org/abs/1910.13413 · HTML: https://mlsanity.com/paper/1910.13413/

## Key facts

- Cited by 167 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2019.

## Built on by (most cited first, 30 of 167)

- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [FastSHAP: Real-Time Shapley Value Estimation](https://mlsanity.com/paper/2107.07436/) (2021)
- [Algorithms to estimate Shapley value feature attributions](https://mlsanity.com/paper/2207.07605/) (2022)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Impossibility Theorems for Feature Attribution](https://mlsanity.com/paper/2212.11870/) (2022)
- [From Shapley Values to Generalized Additive Models and back](https://mlsanity.com/paper/2209.04012/) (2022)
- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022)
- [DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models](https://mlsanity.com/paper/2206.06821/) (2022)
- [Why did the distribution change?](https://mlsanity.com/paper/2102.13384/) (2021)
- [Explaining a Series of Models by Propagating Shapley Values](https://mlsanity.com/paper/2105.00108/) (2021)
- [WeightedSHAP: analyzing and improving Shapley based feature attributions](https://mlsanity.com/paper/2209.13429/) (2022)
- [Explaining Predictive Uncertainty with Information Theoretic Shapley Values](https://mlsanity.com/paper/2306.05724/) (2023)
- [RKHS-SHAP: Shapley Values for Kernel Methods](https://mlsanity.com/paper/2110.09167/) (2021)
- [Fast TreeSHAP: Accelerating SHAP Value Computation for Trees](https://mlsanity.com/paper/2109.09847/) (2021)
- [Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science](https://mlsanity.com/paper/2303.00652/) (2023)
- [On Locality of Local Explanation Models](https://mlsanity.com/paper/2106.14648/) (2021)
- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction](https://mlsanity.com/paper/2110.05304/) (2021)
- [Marrying Fairness and Explainability in Supervised Learning](https://mlsanity.com/paper/2204.02947/) (2022)
- [Provably Accurate Shapley Value Estimation via Leverage Score Sampling](https://mlsanity.com/paper/2410.01917/) (2024)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [Counterfactual Shapley Additive Explanations](https://mlsanity.com/paper/2110.14270/) (2021)
- [groupShapley: Efficient prediction explanation with Shapley values for feature groups](https://mlsanity.com/paper/2106.12228/) (2021)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud](https://mlsanity.com/paper/2109.03285/) (2021)
- [Regression-adjusted Monte Carlo Estimators for Shapley Values and Probabilistic Values](https://mlsanity.com/paper/2506.11849/) (2025)
- [Rational Shapley Values](https://mlsanity.com/paper/2106.10191/) (2021)
- [Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detection](https://mlsanity.com/paper/2205.03302/) (2022)
- [Unifying local and global model explanations by functional decomposition of low dimensional structures](https://mlsanity.com/paper/2208.06151/) (2022)

## Abstract (arXiv)

We discuss promising recent contributions on quantifying feature relevance using Shapley values, where we observed some confusion on which probability distribution is the right one for dropped features. We argue that the confusion is based on not carefully distinguishing between observational and interventional conditional probabilities and try a clarification based on Pearl's seminal work on causality. We conclude that unconditional rather than conditional expectations provide the right notion of dropping features in contradiction to the theoretical justification of the software package SHAP. Parts of SHAP are unaffected because unconditional expectations (which we argue to be conceptually right) are used as approximation for the conditional ones, which encouraged others to `improve' SHAP in a way that we believe to be flawed.