# New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

arXiv: https://arxiv.org/abs/2211.02912 · HTML: https://mlsanity.com/paper/2211.02912/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 60 references that are in the map.
- First posted on arXiv 5 Nov 2022.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [Compositional Explanations of Neurons](https://mlsanity.com/paper/2006.14032/) (2020)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [Abduction-Based Explanations for Machine Learning Models](https://mlsanity.com/paper/1811.10656/) (2018)
- [Explaining Image Classifiers by Counterfactual Generation](https://mlsanity.com/paper/1807.08024/) (2018)
- [Sanity Checks for Saliency Metrics](https://mlsanity.com/paper/1912.01451/) (2019)
- [Restricting the Flow: Information Bottlenecks for Attribution](https://mlsanity.com/paper/2001.00396/) (2020)
- [Interpreting Deep Visual Representations via Network Dissection](https://mlsanity.com/paper/1711.05611/) (2017)
- [Benchmarking Attribution Methods with Relative Feature Importance](https://mlsanity.com/paper/1907.09701/) (2019)
- [How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking](https://mlsanity.com/paper/2004.14992/) (2020)
- [Shapley values for feature selection: The good, the bad, and the axioms](https://mlsanity.com/paper/2102.10936/) (2021)
- [Understanding Individual Decisions of CNNs via Contrastive Backpropagation](https://mlsanity.com/paper/1812.02100/) (2018)

## Built on by (most cited first, 10 of 10)

- [Understanding and Improving Feature Learning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2304.11327/) (2023)
- [Concept-Centric Token Interpretation for Vector-Quantized Generative Models](https://mlsanity.com/paper/2506.00698/) (2025)
- [Time series saliency maps: explaining models across multiple domains](https://mlsanity.com/paper/2505.13100/) (2025)
- [Distance-Aware eXplanation Based Learning](https://mlsanity.com/paper/2309.05548/) (2023)
- [Evaluating Saliency Explanations in NLP by Crowdsourcing](https://mlsanity.com/paper/2405.10767/) (2024)
- [Feature Attribution from First Principles](https://mlsanity.com/paper/2505.24729/) (2025)
- [Harmonizing Feature Attributions Across Deep Learning Architectures: Enhancing Interpretability and Consistency](https://mlsanity.com/paper/2307.02150/) (2023)
- [LIPEx-Locally Interpretable Probabilistic Explanations-To Look Beyond The True Class](https://mlsanity.com/paper/2310.04856/) (2023)
- [Noise-Free Explanation for Driving Action Prediction](https://mlsanity.com/paper/2407.06339/) (2024)
- [Improving Adversarial Robustness of Attribution via Implicit Regularization](https://mlsanity.com/paper/2605.29983/) (2026)

## Abstract (arXiv)

Saliency methods compute heat maps that highlight portions of an input that were most em important for the label assigned to it by a deep net. Evaluations of saliency methods convert this heat map into a new em masked input by retaining the k highest-ranked pixels of the original input and replacing the rest with textquotedblleft uninformativetextquotedblright pixels, and checking if the net's output is mostly unchanged. This is usually seen as an em explanation of the output, but the current paper highlights reasons why this inference of causality may be suspect. Inspired by logic concepts of em completeness & soundness, it observes that the above type of evaluation focuses on completeness of the explanation, but ignores soundness. New evaluation metrics are introduced to capture both notions, while staying in an em intrinsic framework -- i.e., using the dataset and the net, but no separately trained nets, human evaluations, etc. A simple saliency method is described that matches or outperforms prior methods in the evaluations. Experiments also suggest new intrinsic justifications, based on soundness, for popular heuristic tricks such as TV regularization and upsampling.