# Explanation Multiplicity in SHAP: Characterization and Assessment

arXiv: https://arxiv.org/abs/2601.12654 · HTML: https://mlsanity.com/paper/2601.12654/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 19 Jan 2026.

## Builds on (most cited first, 24 of 24)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [True to the Model or True to the Data?](https://mlsanity.com/paper/2006.16234/) (2020)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models](https://mlsanity.com/paper/2001.11757/) (2020)
- [Provably Accurate Shapley Value Estimation via Leverage Score Sampling](https://mlsanity.com/paper/2410.01917/) (2024)
- [Setting the Right Expectations: Algorithmic Recourse Over Time](https://mlsanity.com/paper/2309.06969/) (2023)
- [How should AI decisions be explained? Requirements for Explanations from the Perspective of European Law](https://mlsanity.com/paper/2404.12762/) (2024)
- [An empirical study of the effect of background data size on the stability of SHapley Additive exPlanations (SHAP) for deep learning models](https://mlsanity.com/paper/2204.11351/) (2022)
- [Efficient and Accurate Explanation Estimation with Distribution Compression](https://mlsanity.com/paper/2406.18334/) (2024)
- [SHAP-based Explanations are Sensitive to Feature Representation](https://mlsanity.com/paper/2505.08345/) (2025)
- [Robust Explanations Through Uncertainty Decomposition: A Path to Trustworthier AI](https://mlsanity.com/paper/2507.12913/) (2025)

## Built on by (most cited first, 5 of 5)

- [ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations](https://mlsanity.com/paper/2601.23068/) (2026)
- [Hypothesis Class Determines Explanation: Why Accurate Models Disagree on Feature Attribution](https://mlsanity.com/paper/2603.15821/) (2026)
- [First-Mover Bias in Gradient Boosting Explanations: Mechanism, Detection, and Resolution](https://mlsanity.com/paper/2603.22346/) (2026)
- [Model Stealing Through the Lens of Model Multiplicity](https://mlsanity.com/paper/2606.15493/) (2026)
- [RelShap: Relationally Consistent Shapley Explanations](https://mlsanity.com/paper/2608.11508/) (2026)

## Abstract (arXiv)

SHAP explanations are widely used in high-stakes settings to justify decisions, yet they can differ substantially across repeated runs, even when the model, the input instance, and the prediction are held fixed. Prior work has documented disagreement between explanation methods; we show that substantial disagreement arises even within SHAP across reruns of the same estimator on the same trained model and instance. We call this phenomenon explanation multiplicity and develop an evaluation methodology for characterizing it under deployment-realistic computational budgets, combining a dual-seed protocol that compares model-induced and explainer-induced variability, a hierarchy of magnitude-based, rank-based, and set-based metrics, and randomized Dirichlet and Mallows null models that provide reference scales for observed disagreement. Across multiple datasets, models, and sampling strategies, we find that explanation multiplicity is pervasive and persists even for high-confidence predictions. The relative contribution of each source depends on the data regime and model: model-induced disagreement is generally greater on smaller datasets, while explainer-induced disagreement is greater on larger datasets. Commonly used L2 distance can understate this instability, while rank-based metrics reveal substantial changes in top-ranked features, including the leading feature. Improved sampling methods such as CTE do not eliminate rank-level multiplicity, and K-Means reduces run-to-run variation while its compressed-background explanations can diverge from the empirical-distribution reference. Practitioners should treat single-run SHAP outputs as realizations of a distribution rather than as authoritative artifacts.