# On the Robustness of Removal-Based Feature Attributions

arXiv: https://arxiv.org/abs/2306.07462 · HTML: https://mlsanity.com/paper/2306.07462/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 64 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 30 of 42)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks](https://mlsanity.com/paper/1702.01135/) (2017)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [Explanations can be manipulated and geometry is to blame](https://mlsanity.com/paper/1906.07983/) (2019)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Spectral Norm Regularization for Improving the Generalizability of Deep Learning](https://mlsanity.com/paper/1705.10941/) (2017)
- [Semidefinite relaxations for certifying robustness to adversarial examples](https://mlsanity.com/paper/1811.01057/) (2018)
- [Algorithms to estimate Shapley value feature attributions](https://mlsanity.com/paper/2207.07605/) (2022)
- [Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability](https://mlsanity.com/paper/1910.06358/) (2019)
- [Fooling Neural Network Interpretations via Adversarial Model Manipulation](https://mlsanity.com/paper/1902.02041/) (2019)

## Built on by (most cited first, 19 of 19)

- [shapiq: Shapley Interactions for Machine Learning](https://mlsanity.com/paper/2410.01649/) (2024)
- [Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution](https://mlsanity.com/paper/2401.15866/) (2024)
- [Probabilistic Stability Guarantees for Feature Attributions](https://mlsanity.com/paper/2504.13787/) (2025)
- [Provably Better Explanations with Optimized Aggregation of Feature Attributions](https://mlsanity.com/paper/2406.05090/) (2024)
- [Efficient and Accurate Explanation Estimation with Distribution Compression](https://mlsanity.com/paper/2406.18334/) (2024)
- [Pixel-level Certified Explanations via Randomized Smoothing](https://mlsanity.com/paper/2506.15499/) (2025)
- [Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibration](https://mlsanity.com/paper/2511.10439/) (2025)
- [On the Robustness of Global Feature Effect Explanations](https://mlsanity.com/paper/2406.09069/) (2024)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [Assessing biomedical knowledge robustness in large language models by query-efficient sampling attacks](https://mlsanity.com/paper/2402.10527/) (2024)
- [Error Analysis of Shapley Value-Based Model Explanations: An Informative Perspective](https://mlsanity.com/paper/2404.13522/) (2024)
- [Fast Fourier Correlation is a Highly Efficient and Accurate Feature Attribution Algorithm from the Perspective of Control Theory and Game Theory](https://mlsanity.com/paper/2504.02016/) (2025)
- [Bound Propagation meets Constraint Simplification: Improving Logic-based XAI for Neural Networks](https://mlsanity.com/paper/2603.01923/) (2026)
- [Missingness Bias Calibration in Feature Attribution Explanations](https://mlsanity.com/paper/2603.04831/) (2026)
- [DePICT: Decision-Preserving Interface for Constrained Downstream Tasks](https://mlsanity.com/paper/2610.03945/) (2026)
- [Comprehensive and Reliable Feature Attribution for Diverse Modalities and Models via Frequency-Domain Insights](https://mlsanity.com/paper/2411.18343/) (2024)
- [Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations](https://mlsanity.com/paper/2506.19630/) (2025)
- [Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift](https://mlsanity.com/paper/2605.21849/) (2026)
- [Improving Adversarial Robustness of Attribution via Implicit Regularization](https://mlsanity.com/paper/2605.29983/) (2026)

## Abstract (arXiv)

To explain predictions made by complex machine learning models, many feature attribution methods have been developed that assign importance scores to input features. Some recent work challenges the robustness of these methods by showing that they are sensitive to input and model perturbations, while other work addresses this issue by proposing robust attribution methods. However, previous work on attribution robustness has focused primarily on gradient-based feature attributions, whereas the robustness of removal-based attribution methods is not currently well understood. To bridge this gap, we theoretically characterize the robustness properties of removal-based feature attributions. Specifically, we provide a unified analysis of such methods and derive upper bounds for the difference between intact and perturbed attributions, under settings of both input and model perturbations. Our empirical results on synthetic and real-world data validate our theoretical results and demonstrate their practical implications, including the ability to increase attribution robustness by improving the model's Lipschitz regularity.