# Revisiting Sanity Checks for Saliency Maps

arXiv: https://arxiv.org/abs/2110.14297 · HTML: https://mlsanity.com/paper/2110.14297/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 19 references that are in the map.
- First posted on arXiv 27 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Debugging Tests for Model Explanations](https://mlsanity.com/paper/2011.05429/) (2020)
- [Sanity Checks for Saliency Metrics](https://mlsanity.com/paper/1912.01451/) (2019)
- [Do Feature Attribution Methods Correctly Attribute Features?](https://mlsanity.com/paper/2104.14403/) (2021)
- [Evaluating Saliency Methods for Neural Language Models](https://mlsanity.com/paper/2104.05824/) (2021)
- [How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networks](https://mlsanity.com/paper/2009.04521/) (2020)
- [Assessing the validity of saliency maps for abnormality localization in medical imaging](https://mlsanity.com/paper/2006.00063/) (2020)

## Built on by (most cited first, 20 of 20)

- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus](https://mlsanity.com/paper/2302.07265/) (2023)
- [Shortcomings of Top-Down Randomization-Based Sanity Checks for Evaluations of Deep Neural Network Explanations](https://mlsanity.com/paper/2211.12486/) (2022)
- [Evaluating Feature Attribution Methods in the Image Domain](https://mlsanity.com/paper/2202.12270/) (2022)
- [Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Test](https://mlsanity.com/paper/2401.06465/) (2024)
- [A Fresh Look at Sanity Checks for Saliency Maps](https://mlsanity.com/paper/2405.02383/) (2024)
- [Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluations](https://mlsanity.com/paper/2406.05068/) (2024)
- [Generalizing Backpropagation for Gradient-Based Interpretability](https://mlsanity.com/paper/2307.03056/) (2023)
- [Don't Get Me Wrong: How to Apply Deep Visual Interpretations to Time Series](https://mlsanity.com/paper/2203.07861/) (2022)
- [Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization](https://mlsanity.com/paper/2408.04983/) (2024)
- [eXpLogic: Explaining Logic Types and Patterns in DiffLogic Networks](https://mlsanity.com/paper/2503.09910/) (2025)
- [Sanity Checks for Saliency Methods Explaining Object Detectors](https://mlsanity.com/paper/2306.02424/) (2023)
- [Scalable Concept Extraction in Industry 4.0](https://mlsanity.com/paper/2306.03551/) (2023)
- [Is Task-Agnostic Explainable AI a Myth?](https://mlsanity.com/paper/2307.06963/) (2023)
- [Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry](https://mlsanity.com/paper/2412.20798/) (2024)
- [RoentMod: A Synthetic Chest X-Ray Modification Model to Identify and Correct Image Interpretation Model Shortcuts](https://mlsanity.com/paper/2509.08640/) (2025)
- [Explaining, Analyzing, and Probing Representations of Self-Supervised Learning Models for Sensor-based Human Activity Recognition](https://mlsanity.com/paper/2304.07304/) (2023)
- [Saliency Methods are Encoders: Analysing Logical Relations Towards Interpretation](https://mlsanity.com/paper/2412.16204/) (2024)
- [Unifying Perplexing Behaviors in Modified BP Attributions through Alignment Perspective](https://mlsanity.com/paper/2503.11160/) (2025)
- [Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning](https://mlsanity.com/paper/2511.20591/) (2025)

## Abstract (arXiv)

Saliency methods are a popular approach for model debugging and explainability. However, in the absence of ground-truth data for what the correct maps should be, evaluating and comparing different approaches remains a long-standing challenge. The sanity checks methodology of Adebayo et al [Neurips 2018] has sought to address this challenge. They argue that some popular saliency methods should not be used for explainability purposes since the maps they produce are not sensitive to the underlying model that is to be explained. Through a causal re-framing of their objective, we argue that their empirical evaluation does not fully establish these conclusions, due to a form of confounding introduced by the tasks they evaluate on. Through various experiments on simple custom tasks we demonstrate that some of their conclusions may indeed be artifacts of the tasks more than a criticism of the saliency methods themselves. More broadly, our work challenges the utility of the sanity check methodology, and further highlights that saliency map evaluation beyond ad-hoc visual examination remains a fundamental challenge.