# An unexpected unity among methods for interpreting model predictions

arXiv: https://arxiv.org/abs/1611.07478 · HTML: https://mlsanity.com/paper/1611.07478/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2016.

## Built on by (most cited first, 17 of 17)

- [Explainable Artificial Intelligence Approaches: A Survey](https://mlsanity.com/paper/2101.09429/) (2021)
- [Augmenting Interpretable Models with LLMs during Training](https://mlsanity.com/paper/2209.11799/) (2022)
- [Machine Learning for a Sustainable Energy Future](https://mlsanity.com/paper/2210.10391/) (2022)
- [Decorrelated Variable Importance](https://mlsanity.com/paper/2111.10853/) (2021)
- [Comprehensible Convolutional Neural Networks via Guided Concept Learning](https://mlsanity.com/paper/2101.03919/) (2021)
- [Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Model](https://mlsanity.com/paper/2110.03184/) (2021)
- [Detection of ADHD based on Eye Movements during Natural Viewing](https://mlsanity.com/paper/2207.01377/) (2022)
- [Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insights](https://mlsanity.com/paper/2208.06096/) (2022)
- [ShaRP: Explaining Rankings and Preferences with Shapley Values](https://mlsanity.com/paper/2401.16744/) (2024)
- [Mapping Knowledge Representations to Concepts: A Review and New Perspectives](https://mlsanity.com/paper/2301.00189/) (2022)
- [AI-Augmented LLMs Achieve Therapist-Level Responses in Motivational Interviewing](https://mlsanity.com/paper/2505.17380/) (2025)
- [Information-theoretic Evolution of Model Agnostic Global Explanations](https://mlsanity.com/paper/2105.06956/) (2021)
- [Easy to Decide, Hard to Agree: Reducing Disagreements Between Saliency Methods](https://mlsanity.com/paper/2211.08369/) (2022)
- [Model-Agnostic Interpretation Framework in Machine Learning: A Comparative Study in NBA Sports](https://mlsanity.com/paper/2401.02630/) (2024)
- [Adversarial Evasion Attacks on Computer Vision using SHAP Values](https://mlsanity.com/paper/2601.10587/) (2026)
- [Wavelet Scattering Transform for Interpretable Schizophrenia Biomarker Discovery and Classification from Resting-State EEG](https://mlsanity.com/paper/2607.05282/) (2026)
- [ChatImage: Navigating Long-Form LLM Answers through Interactive Images](https://mlsanity.com/paper/2607.05290/) (2026)

## Abstract (arXiv)

Understanding why a model made a certain prediction is crucial in many data science fields. Interpretable predictions engender appropriate trust and provide insight into how the model may be improved. However, with large modern datasets the best accuracy is often achieved by complex models even experts struggle to interpret, which creates a tension between accuracy and interpretability. Recently, several methods have been proposed for interpreting predictions from complex models by estimating the importance of input features. Here, we present how a model-agnostic additive representation of the importance of input features unifies current methods. This representation is optimal, in the sense that it is the only set of additive values that satisfies important properties. We show how we can leverage these properties to create novel visual explanations of model predictions. The thread of unity that this representation weaves through the literature indicates that there are common principles to be learned about the interpretation of model predictions that apply in many scenarios.