# PredDiff: Explanations and Interactions from Conditional Expectations

arXiv: https://arxiv.org/abs/2102.13519 · HTML: https://mlsanity.com/paper/2102.13519/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 67 references that are in the map.
- First posted on arXiv 26 Feb 2021.

## Builds on (most cited first, 30 of 36)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Visualizing Deep Neural Network Decisions: Prediction Difference Analysis](https://mlsanity.com/paper/1702.04595/) (2017)
- [The Shapley Taylor Interaction Index](https://mlsanity.com/paper/1902.05622/) (2019)
- [Higher-Order Explanations of Graph Neural Networks via Relevant Walks](https://mlsanity.com/paper/2006.03589/) (2020)
- [Explaining Explanations: Axiomatic Feature Interactions for Deep Networks](https://mlsanity.com/paper/2002.04138/) (2020)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [How does this interaction affect me? Interpretable attribution for feature interactions](https://mlsanity.com/paper/2006.10965/) (2020)
- [Fairwashing Explanations with Off-Manifold Detergent](https://mlsanity.com/paper/2007.09969/) (2020)
- [Visualizing the Feature Importance for Black Box Models](https://mlsanity.com/paper/1804.06620/) (2018)
- [Interpretation of NLP models through input marginalization](https://mlsanity.com/paper/2010.13984/) (2020)
- [Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Models](https://mlsanity.com/paper/1911.04974/) (2019)
- [Building and Interpreting Deep Similarity Models](https://mlsanity.com/paper/2003.05431/) (2020)
- [Feature Removal Is a Unifying Principle for Model Explanation Methods](https://mlsanity.com/paper/2011.03623/) (2020)
- [Domain aware medical image classifier interpretation by counterfactual impact analysis](https://mlsanity.com/paper/2007.06312/) (2020)

## Built on by (most cited first, 17 of 17)

- [From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation](https://mlsanity.com/paper/2206.03208/) (2022)
- [Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees](https://mlsanity.com/paper/2301.11911/) (2023)
- [Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks](https://mlsanity.com/paper/2401.06654/) (2024)
- [Unifying Feature-Based Explanations with Functional ANOVA and Cooperative Game Theory](https://mlsanity.com/paper/2412.17152/) (2024)
- [Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features](https://mlsanity.com/paper/2408.17198/) (2024)
- [Identifying Important Group of Pixels using Interactions](https://mlsanity.com/paper/2401.03785/) (2024)
- [Explainable AI needs formalization](https://mlsanity.com/paper/2409.14590/) (2024)
- [Sparse Subspace Clustering for Concept Discovery (SSCCD)](https://mlsanity.com/paper/2203.06043/) (2022)
- [An XAI framework for robust and transparent data-driven wind turbine power curve models](https://mlsanity.com/paper/2304.09835/) (2023)
- [Explaining Time Series Classification Predictions via Causal Attributions](https://mlsanity.com/paper/2405.15871/) (2024)
- [Explaining Object Detectors via Collective Contribution of Pixels](https://mlsanity.com/paper/2412.00666/) (2024)
- [Distributing Synergy Functions: Unifying Game-Theoretic Interaction Methods for Machine-Learning Explainability](https://mlsanity.com/paper/2305.03100/) (2023)
- [Insights Into the Inner Workings of Transformer Models for Protein Function Prediction](https://mlsanity.com/paper/2309.03631/) (2023)
- [Minimizing False-Positive Attributions in Explanations of Non-Linear Models](https://mlsanity.com/paper/2505.11210/) (2025)
- [H-Sets: Hessian-Guided Discovery of Set-Level Feature Interactions in Image Classifiers](https://mlsanity.com/paper/2604.22045/) (2026)
- [Explaining Temporal Graph Neural Networks via Feature-induced Information Flow](https://mlsanity.com/paper/2606.27201/) (2026)
- [Rethinking Least-Core Computation in Contextual-Distractor Games](https://mlsanity.com/paper/2610.06087/) (2026)

## Abstract (arXiv)

PredDiff is a model-agnostic, local attribution method that is firmly rooted in probability theory. Its simple intuition is to measure prediction changes while marginalizing features. In this work, we clarify properties of PredDiff and its close connection to Shapley values. We stress important differences between classification and regression, which require a specific treatment within both formalisms. We extend PredDiff by introducing a new, well-founded measure for interaction effects between arbitrary feature subsets. The study of interaction effects represents an inevitable step towards a comprehensive understanding of black-box models and is particularly important for science applications. Equipped with our novel interaction measure, PredDiff is a promising model-agnostic approach for obtaining reliable, numerically inexpensive and theoretically sound attributions.