# The Explanation Game: Explaining Machine Learning Models Using Shapley Values

arXiv: https://arxiv.org/abs/1909.08128 · HTML: https://mlsanity.com/paper/1909.08128/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2019.

## Built on by (most cited first, 8 of 8)

- [Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals](https://mlsanity.com/paper/2103.11972/) (2021)
- [Shapley values for feature selection: The good, the bad, and the axioms](https://mlsanity.com/paper/2102.10936/) (2021)
- [Interpretable Data-Based Explanations for Fairness Debugging](https://mlsanity.com/paper/2112.09745/) (2021)
- [How should AI decisions be explained? Requirements for Explanations from the Perspective of European Law](https://mlsanity.com/paper/2404.12762/) (2024)
- [AI Data Readiness Inspector (AIDRIN) for Quantitative Assessment of Data Readiness for AI](https://mlsanity.com/paper/2406.19256/) (2024)
- [Selective Intervention Planning using Restless Multi-Armed Bandits to Improve Maternal and Child Health Outcomes](https://mlsanity.com/paper/2103.09052/) (2021)
- [A general-purpose method for applying Explainable AI for Anomaly Detection](https://mlsanity.com/paper/2207.11564/) (2022)
- [Beyond the Black Box: Do More Complex Deep Learning Models Provide Superior XAI Explanations?](https://mlsanity.com/paper/2405.08658/) (2024)

## Abstract (arXiv)

A number of techniques have been proposed to explain a machine learning model's prediction by attributing it to the corresponding input features. Popular among these are techniques that apply the Shapley value method from cooperative game theory. While existing papers focus on the axiomatic motivation of Shapley values, and efficient techniques for computing them, they offer little justification for the game formulations used, and do not address the uncertainty implicit in their methods' outputs. For instance, the popular SHAP algorithm's formulation may give substantial attributions to features that play no role in the model. In this work, we illustrate how subtle differences in the underlying game formulations of existing methods can cause large differences in the attributions for a prediction. We then present a general game formulation that unifies existing methods, and enables straightforward confidence intervals on their attributions. Furthermore, it allows us to interpret the attributions as contrastive explanations of an input relative to a distribution of reference inputs. We tie this idea to classic research in cognitive psychology on contrastive explanations, and propose a conceptual framework for generating and interpreting explanations for ML models, called formulate, approximate, explain (FAE). We apply this framework to explain black-box models trained on two UCI datasets and a Lending Club dataset.