# Visualizing Deep Networks by Optimizing with Integrated Gradients

arXiv: https://arxiv.org/abs/1905.00954 · HTML: https://mlsanity.com/paper/1905.00954/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2019.

## Built on by (most cited first, 30 of 59)

- [Impossibility Theorems for Feature Attribution](https://mlsanity.com/paper/2212.11870/) (2022)
- [Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2412.02104/) (2024)
- [Counterfactual State Explanations for Reinforcement Learning Agents via Generative Deep Learning](https://mlsanity.com/paper/2101.12446/) (2021)
- [Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/2103.13859/) (2021)
- [Gradient based Feature Attribution in Explainable AI: A Technical Review](https://mlsanity.com/paper/2403.10415/) (2024)
- [Neural Response Interpretation through the Lens of Critical Pathways](https://mlsanity.com/paper/2103.16886/) (2021)
- [Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution](https://mlsanity.com/paper/2108.01070/) (2021)
- [Do Explanations Explain? Model Knows Best](https://mlsanity.com/paper/2203.02269/) (2022)
- [Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset](https://mlsanity.com/paper/2103.10005/) (2021)
- [Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing](https://mlsanity.com/paper/2410.07054/) (2024)
- [Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Information](https://mlsanity.com/paper/2110.01471/) (2021)
- [IDGI: A Framework to Eliminate Explanation Noise from Integrated Gradients](https://mlsanity.com/paper/2303.14242/) (2023)
- [ECLAD: Extracting Concepts with Local Aggregated Descriptors](https://mlsanity.com/paper/2206.04531/) (2022)
- [Where do Large Vision-Language Models Look at when Answering Questions?](https://mlsanity.com/paper/2503.13891/) (2025)
- [Deletion and Insertion Tests in Regression Models](https://mlsanity.com/paper/2205.12423/) (2022)
- [Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods](https://mlsanity.com/paper/2212.06872/) (2022)
- [A Coupled Design of Exploiting Record Similarity for Practical Vertical Federated Learning](https://mlsanity.com/paper/2106.06312/) (2021)
- [A Psychological Theory of Explainability](https://mlsanity.com/paper/2205.08452/) (2022)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [On the Importance of Domain-specific Explanations in AI-based Cybersecurity Systems (Technical Report)](https://mlsanity.com/paper/2108.02006/) (2021)
- ["Is your explanation stable?": A Robustness Evaluation Framework for Feature Attribution](https://mlsanity.com/paper/2209.01782/) (2022)
- [Individual Explanations in Machine Learning Models: A Survey for Practitioners](https://mlsanity.com/paper/2104.04144/) (2021)
- [Multi-Stage Prototype Learning for Interpretable Time Series Classification](https://mlsanity.com/paper/2106.09636/) (2021)
- [EvdCLIP: Improving Vision-Language Retrieval with Entity Visual Descriptions from Large Language Models](https://mlsanity.com/paper/2505.18594/) (2025)
- [CEIR: Concept-based Explainable Image Representation Learning](https://mlsanity.com/paper/2312.10747/) (2023)
- [Does Dataset Complexity Matters for Model Explainers?](https://mlsanity.com/paper/2107.02661/) (2021)
- [From Heatmaps to Structural Explanations of Image Classifiers](https://mlsanity.com/paper/2109.06365/) (2021)
- [Precise Benchmarking of Explainable AI Attribution Methods](https://mlsanity.com/paper/2308.03161/) (2023)
- [Spatio-Temporal Perturbations for Video Attribution](https://mlsanity.com/paper/2109.00222/) (2021)
- [Maximum Entropy Baseline for Integrated Gradients](https://mlsanity.com/paper/2204.05948/) (2022)

## Abstract (arXiv)

Understanding and interpreting the decisions made by deep learning models is valuable in many domains. In computer vision, computing heatmaps from a deep network is a popular approach for visualizing and understanding deep networks. However, heatmaps that do not correlate with the network may mislead human, hence the performance of heatmaps in providing a faithful explanation to the underlying deep network is crucial. In this paper, we propose I-GOS, which optimizes for a heatmap so that the classification scores on the masked image would maximally decrease. The main novelty of the approach is to compute descent directions based on the integrated gradients instead of the normal gradient, which avoids local optima and speeds up convergence. Compared with previous approaches, our method can flexibly compute heatmaps at any resolution for different user needs. Extensive experiments on several benchmark datasets show that the heatmaps produced by our approach are more correlated with the decision of the underlying deep network, in comparison with other state-of-the-art approaches.