# The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets

arXiv: https://arxiv.org/abs/2009.11023 · HTML: https://mlsanity.com/paper/2009.11023/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2020.

## Built on by (most cited first, 14 of 14)

- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks](https://mlsanity.com/paper/2105.03761/) (2021)
- ["Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification](https://mlsanity.com/paper/2111.07367/) (2021)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Zorro: Valid, Sparse, and Stable Explanations in Graph Neural Networks](https://mlsanity.com/paper/2105.08621/) (2021)
- [The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models](https://mlsanity.com/paper/2404.03189/) (2024)
- [Probabilistic Sufficient Explanations](https://mlsanity.com/paper/2105.10118/) (2021)
- [Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations](https://mlsanity.com/paper/2106.13876/) (2021)
- [Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features](https://mlsanity.com/paper/2111.03740/) (2021)
- [Interactive Prompt Debugging with Sequence Salience](https://mlsanity.com/paper/2404.07498/) (2024)
- [Explainability's Gain is Optimality's Loss? -- How Explanations Bias Decision-making](https://mlsanity.com/paper/2206.08705/) (2022)
- [Interpretability in Symbolic Regression: a benchmark of Explanatory Methods using the Feynman data set](https://mlsanity.com/paper/2404.05908/) (2024)
- [SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations](https://mlsanity.com/paper/2305.13235/) (2023)
- [CAREF: Calibration-Aware Regularization for Explanation Faithfulness Without Rationale Supervision](https://mlsanity.com/paper/2605.27835/) (2026)

## Abstract (arXiv)

For neural models to garner widespread public trust and ensure fairness, we must have human-intelligible explanations for their predictions. Recently, an increasing number of works focus on explaining the predictions of neural models in terms of the relevance of the input features. In this work, we show that feature-based explanations pose problems even for explaining trivial models. We show that, in certain cases, there exist at least two ground-truth feature-based explanations, and that, sometimes, neither of them is enough to provide a complete view of the decision-making process of the model. Moreover, we show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations, despite the apparently implicit assumption that explainers should look for one specific feature-based explanation. These findings bring an additional dimension to consider in both developing and choosing explainers.