# Feature Removal Is a Unifying Principle for Model Explanation Methods

arXiv: https://arxiv.org/abs/2011.03623 · HTML: https://mlsanity.com/paper/2011.03623/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2020.

## Built on by (most cited first, 21 of 21)

- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Reinforced Causal Explainer for Graph Neural Networks](https://mlsanity.com/paper/2204.11028/) (2022)
- [Contrastive Video Question Answering via Video Graph Transformer](https://mlsanity.com/paper/2302.13668/) (2023)
- [PredDiff: Explanations and Interactions from Conditional Expectations](https://mlsanity.com/paper/2102.13519/) (2021)
- [Explainability in Music Recommender Systems](https://mlsanity.com/paper/2201.10528/) (2022)
- [Explanations of Black-Box Models based on Directional Feature Interactions](https://mlsanity.com/paper/2304.07670/) (2023)
- [Towards Rigorous Interpretations: a Formalisation of Feature Attribution](https://mlsanity.com/paper/2104.12437/) (2021)
- [Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?](https://mlsanity.com/paper/2105.10719/) (2021)
- [Deconfounding to Explanation Evaluation in Graph Neural Networks](https://mlsanity.com/paper/2201.08802/) (2022)
- [How explainable are adversarially-robust CNNs?](https://mlsanity.com/paper/2205.13042/) (2022)
- [Prototype-Based Interpretability for Legal Citation Prediction](https://mlsanity.com/paper/2305.16490/) (2023)
- [Attribution of Predictive Uncertainties in Classification Models](https://mlsanity.com/paper/2107.08756/) (2021)
- [Shapley variable importance clouds for interpretable machine learning](https://mlsanity.com/paper/2110.02484/) (2021)
- [Boundary-Aware Uncertainty for Feature Attribution Explainers](https://mlsanity.com/paper/2210.02419/) (2022)
- [Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models?](https://mlsanity.com/paper/2110.11929/) (2021)
- [SmoothHess: ReLU Network Feature Interactions via Stein's Lemma](https://mlsanity.com/paper/2311.00858/) (2023)
- [An Efficient Framework for Crediting Data Contributors of Diffusion Models](https://mlsanity.com/paper/2407.03153/) (2024)
- [Towards Faithful Knowledge Graph Explanation Through Deep Alignment in Commonsense Question Answering](https://mlsanity.com/paper/2310.04910/) (2023)
- [CAGE: Causality-Aware Shapley Value for Global Explanations](https://mlsanity.com/paper/2404.11208/) (2024)
- [On the Connection between Game-Theoretic Feature Attributions and Counterfactual Explanations](https://mlsanity.com/paper/2307.06941/) (2023)
- [A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions](https://mlsanity.com/paper/2607.08776/) (2026)

## Abstract (arXiv)

Researchers have proposed a wide variety of model explanation approaches, but it remains unclear how most methods are related or when one method is preferable to another. We examine the literature and find that many methods are based on a shared principle of explaining by removing - essentially, measuring the impact of removing sets of features from a model. These methods vary in several respects, so we develop a framework for removal-based explanations that characterizes each method along three dimensions: 1) how the method removes features, 2) what model behavior the method explains, and 3) how the method summarizes each feature's influence. Our framework unifies 26 existing methods, including several of the most widely used approaches (SHAP, LIME, Meaningful Perturbations, permutation tests). Exposing the fundamental similarities between these methods empowers users to reason about which tools to use, and suggests promising directions for ongoing model explainability research.