# Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps

arXiv: https://arxiv.org/abs/1902.04893 · HTML: https://mlsanity.com/paper/1902.04893/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2019.

## Built on by (most cited first, 30 of 34)

- [Do Input Gradients Highlight Discriminative Features?](https://mlsanity.com/paper/2102.12781/) (2021)
- [Gradient based Feature Attribution in Explainable AI: A Technical Review](https://mlsanity.com/paper/2403.10415/) (2024)
- [Surrogate Model-Based Explainability Methods for Point Cloud NNs](https://mlsanity.com/paper/2107.13459/) (2021)
- [New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound](https://mlsanity.com/paper/2211.02912/) (2022)
- [Causal Analysis for Robust Interpretability of Neural Networks](https://mlsanity.com/paper/2305.08950/) (2023)
- [Fidelity of Interpretability Methods and Perturbation Artifacts in Neural Networks](https://mlsanity.com/paper/2203.02928/) (2022)
- [Structured Gradient-based Interpretations via Norm-Regularized Adversarial Training](https://mlsanity.com/paper/2404.04647/) (2024)
- [Evaluation of Interpretability for Deep Learning algorithms in EEG Emotion Recognition: A case study in Autism](https://mlsanity.com/paper/2111.13208/) (2021)
- [Deeply Explain CNN via Hierarchical Decomposition](https://mlsanity.com/paper/2201.09205/) (2022)
- [Uncertainty Propagation in XAI: A Comparison of Analytical and Empirical Estimators](https://mlsanity.com/paper/2504.03736/) (2025)
- [Expected Grad-CAM: Towards gradient faithfulness](https://mlsanity.com/paper/2406.01274/) (2024)
- [Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements](https://mlsanity.com/paper/2206.10587/) (2022)
- [On Pitfalls of textitRemOve-And-Retrain: Data Processing Inequality Perspective](https://mlsanity.com/paper/2304.13836/) (2023)
- [Saliency strikes back: How filtering out high frequencies improves white-box explanations](https://mlsanity.com/paper/2307.09591/) (2023)
- [Evaluation of importance estimators in deep learning classifiers for Computed Tomography](https://mlsanity.com/paper/2209.15398/) (2022)
- [Towards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations](https://mlsanity.com/paper/2306.13971/) (2023)
- [CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Models](https://mlsanity.com/paper/2404.14830/) (2024)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [On the Complexity-Faithfulness Trade-off of Gradient-Based Explanations](https://mlsanity.com/paper/2508.10490/) (2025)
- [On Spectral Properties of Gradient-based Explanation Methods](https://mlsanity.com/paper/2508.10595/) (2025)
- [Understanding CNNs from excitations](https://mlsanity.com/paper/2205.00932/) (2022)
- [Bidirectional Emergent Language in Situated Environments](https://mlsanity.com/paper/2408.14649/) (2024)
- [Gaussian Smoothing in Saliency Maps: The Stability-Fidelity Trade-Off in Neural Network Interpretability](https://mlsanity.com/paper/2411.05837/) (2024)
- [AutoSIGHT: Automatic Eye Tracking-based System for Immediate Grading of Human experTise](https://mlsanity.com/paper/2508.01015/) (2025)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [Attribution Mask: Filtering Out Irrelevant Features By Recursively Focusing Attention on Inputs of DNNs](https://mlsanity.com/paper/2102.07332/) (2021)
- [Deep Learning Image Recognition for Non-images](https://mlsanity.com/paper/2106.14350/) (2021)
- [Unifying Perplexing Behaviors in Modified BP Attributions through Alignment Perspective](https://mlsanity.com/paper/2503.11160/) (2025)
- [Query Circuits: Explaining How Language Models Answer User Prompts](https://mlsanity.com/paper/2509.24808/) (2025)
- [Explore the Ideology of Deep Learning in ENSO Forecasts](https://mlsanity.com/paper/2601.02050/) (2026)

## Abstract (arXiv)

Saliency Map, the gradient of the score function with respect to the input, is the most basic technique for interpreting deep neural network decisions. However, saliency maps are often visually noisy. Although several hypotheses were proposed to account for this phenomenon, there are few works that provide rigorous analyses of noisy saliency maps. In this paper, we firstly propose a new hypothesis that noise may occur in saliency maps when irrelevant features pass through ReLU activation functions. Then, we propose Rectified Gradient, a method that alleviates this problem through layer-wise thresholding during backpropagation. Experiments with neural networks trained on CIFAR-10 and ImageNet showed effectiveness of our method and its superiority to other attribution methods.