# Visual Explanations from Deep Networks via Riemann-Stieltjes Integrated Gradient-based Localization

arXiv: https://arxiv.org/abs/2205.10900 · HTML: https://mlsanity.com/paper/2205.10900/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 22 May 2022.

## Builds on (most cited first, 13 of 13)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [How Important Is a Neuron?](https://mlsanity.com/paper/1805.12233/) (2018)
- [Adapting Grad-CAM for Embedding Networks](https://mlsanity.com/paper/2001.06538/) (2020)
- [Computationally Efficient Measures of Internal Neuron Importance](https://mlsanity.com/paper/1807.09946/) (2018)
- [Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoring](https://mlsanity.com/paper/2102.07805/) (2021)

## Built on by (most cited first, 5 of 5)

- [Pre or Post-Softmax Scores in Gradient-based Attribution Methods, What is Best?](https://mlsanity.com/paper/2306.13197/) (2023)
- [A Vulnerability of Attribution Methods Using Pre-Softmax Scores](https://mlsanity.com/paper/2307.03305/) (2023)
- [Interactive Mars Image Content-Based Search with Interpretable Machine Learning](https://mlsanity.com/paper/2402.16860/) (2024)
- [SCOPE-MRI: Bankart Lesion Detection as a Case Study in Data Curation and Deep Learning for Challenging Diagnoses](https://mlsanity.com/paper/2504.20405/) (2025)
- [Causal Interpretation of Neural Network Computations with Contribution Decomposition](https://mlsanity.com/paper/2603.06557/) (2026)

## Abstract (arXiv)

Neural networks are becoming increasingly better at tasks that involve classifying and recognizing images. At the same time techniques intended to explain the network output have been proposed. One such technique is the Gradient-based Class Activation Map (Grad-CAM), which is able to locate features of an input image at various levels of a convolutional neural network (CNN), but is sensitive to the vanishing gradients problem. There are techniques such as Integrated Gradients (IG), that are not affected by that problem, but its use is limited to the input layer of a network. Here we introduce a new technique to produce visual explanations for the predictions of a CNN. Like Grad-CAM, our method can be applied to any layer of the network, and like Integrated Gradients it is not affected by the problem of vanishing gradients. For efficiency, gradient integration is performed numerically at the layer level using a Riemann-Stieltjes sum approximation. Compared to Grad-CAM, heatmaps produced by our algorithm are better focused in the areas of interest, and their numerical computation is more stable. Our code is available at https://github.com/mlerma54/RSIGradCAM