# Rethinking Positive Aggregation and Propagation of Gradients in Gradient-based Saliency Methods

arXiv: https://arxiv.org/abs/2012.00362 · HTML: https://mlsanity.com/paper/2012.00362/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 7 of 7)

- [Neural Response Interpretation through the Lens of Critical Pathways](https://mlsanity.com/paper/2103.16886/) (2021)
- [Do Explanations Explain? Model Knows Best](https://mlsanity.com/paper/2203.02269/) (2022)
- [Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information](https://mlsanity.com/paper/2110.01471/) (2021)
- [Explaining COVID-19 and Thoracic Pathology Model Predictions by Identifying Informative Input Features](https://mlsanity.com/paper/2104.00411/) (2021)
- [Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Images](https://mlsanity.com/paper/2303.08632/) (2023)
- [Interpretable Vertebral Fracture Diagnosis](https://mlsanity.com/paper/2203.16273/) (2022)
- [Analyzing the Effects of Handling Data Imbalance on Learned Features from Medical Images by Looking Into the Models](https://mlsanity.com/paper/2204.01729/) (2022)

## Abstract (arXiv)

Saliency methods interpret the prediction of a neural network by showing the importance of input elements for that prediction. A popular family of saliency methods utilize gradient information. In this work, we empirically show that two approaches for handling the gradient information, namely positive aggregation, and positive propagation, break these methods. Though these methods reflect visually salient information in the input, they do not explain the model prediction anymore as the generated saliency maps are insensitive to the predicted output and are insensitive to model parameter randomization. Specifically for methods that aggregate the gradients of a chosen layer such as GradCAM++ and FullGrad, exclusively aggregating positive gradients is detrimental. We further support this by proposing several variants of aggregation methods with positive handling of gradient information. For methods that backpropagate gradient information such as LRP, RectGrad, and Guided Backpropagation, we show the destructive effect of exclusively propagating positive gradient information.