# True to the Model or True to the Data?

arXiv: https://arxiv.org/abs/2006.16234 · HTML: https://mlsanity.com/paper/2006.16234/

## Key facts

- Cited by 126 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2020.

## Built on by (most cited first, 30 of 126)

- [Algorithms to estimate Shapley value feature attributions](https://mlsanity.com/paper/2207.07605/) (2022)
- [Faith-Shap: The Faithful Shapley Interaction Index](https://mlsanity.com/paper/2203.00870/) (2022)
- [From Shapley Values to Generalized Additive Models and back](https://mlsanity.com/paper/2209.04012/) (2022)
- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [shapiq: Shapley Interactions for Machine Learning](https://mlsanity.com/paper/2410.01649/) (2024)
- [Explaining a Series of Models by Propagating Shapley Values](https://mlsanity.com/paper/2105.00108/) (2021)
- [Explaining Predictive Uncertainty with Information Theoretic Shapley Values](https://mlsanity.com/paper/2306.05724/) (2023)
- [Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles](https://mlsanity.com/paper/2401.12069/) (2024)
- [RKHS-SHAP: Shapley Values for Kernel Methods](https://mlsanity.com/paper/2110.09167/) (2021)
- [Fast TreeSHAP: Accelerating SHAP Value Computation for Trees](https://mlsanity.com/paper/2109.09847/) (2021)
- [On Locality of Local Explanation Models](https://mlsanity.com/paper/2106.14648/) (2021)
- [Counterfactual Shapley Additive Explanations](https://mlsanity.com/paper/2110.14270/) (2021)
- [Dear XAI Community, We Need to Talk! Fundamental Misconceptions in Current XAI Research](https://mlsanity.com/paper/2306.04292/) (2023)
- [groupShapley: Efficient prediction explanation with Shapley values for feature groups](https://mlsanity.com/paper/2106.12228/) (2021)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Using Shapley Values and Variational Autoencoders to Explain Predictive Models with Dependent Mixed Features](https://mlsanity.com/paper/2111.13507/) (2021)
- [Evaluating Feature Attribution Methods in the Image Domain](https://mlsanity.com/paper/2202.12270/) (2022)
- [Explaining the Uncertain: Stochastic Shapley Values for Gaussian Process Models](https://mlsanity.com/paper/2305.15167/) (2023)
- [Unifying local and global model explanations by functional decomposition of low dimensional structures](https://mlsanity.com/paper/2208.06151/) (2022)
- [Fast Hierarchical Games for Image Explanations](https://mlsanity.com/paper/2104.06164/) (2021)
- [Explainability in Music Recommender Systems](https://mlsanity.com/paper/2201.10528/) (2022)
- [A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Them](https://mlsanity.com/paper/2305.09536/) (2023)
- [Learning to Understand: Identifying Interactions via the M"obius Transform](https://mlsanity.com/paper/2402.02631/) (2024)
- [Statistical Aspects of SHAP: Functional ANOVA for Model Interpretation](https://mlsanity.com/paper/2208.09970/) (2022)
- [Explaining predictive models using Shapley values and non-parametric vine copulas](https://mlsanity.com/paper/2102.06416/) (2021)
- [Accurate Shapley Values for explaining tree-based models](https://mlsanity.com/paper/2106.03820/) (2021)
- [Feature Synergy, Redundancy, and Independence in Global Model Explanations using SHAP Vector Decomposition](https://mlsanity.com/paper/2107.12436/) (2021)
- [Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap](https://mlsanity.com/paper/2201.11676/) (2022)
- [Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams](https://mlsanity.com/paper/2209.01939/) (2022)
- [Stabilizing Estimates of Shapley Values with Control Variates](https://mlsanity.com/paper/2310.07672/) (2023)

## Abstract (arXiv)

A variety of recent papers discuss the application of Shapley values, a concept for explaining coalitional games, for feature attribution in machine learning. However, the correct way to connect a machine learning model to a coalitional game has been a source of controversy. The two main approaches that have been proposed differ in the way that they condition on known features, using either (1) an interventional or (2) an observational conditional expectation. While previous work has argued that one of the two approaches is preferable in general, we argue that the choice is application dependent. Furthermore, we argue that the choice comes down to whether it is desirable to be true to the model or true to the data. We use linear models to investigate this choice. After deriving an efficient method for calculating observational conditional expectation Shapley values for linear models, we investigate how correlation in simulated data impacts the convergence of observational conditional expectation Shapley values. Finally, we present two real data examples that we consider to be representative of possible use cases for feature attribution -- (1) credit risk modeling and (2) biological discovery. We show how a different choice of value function performs better in each scenario, and how possible attributions are impacted by modeling choices.