# A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributions

arXiv: https://arxiv.org/abs/2202.11912 · HTML: https://mlsanity.com/paper/2202.11912/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [The Shattered Gradients Problem: If resnets are the answer, then what is the question?](https://mlsanity.com/paper/1702.08591/) (2017)
- [Guided Integrated Gradients: An Adaptive Path Method for Removing Noise](https://mlsanity.com/paper/2106.09788/) (2021)
- [How Important Is a Neuron?](https://mlsanity.com/paper/1805.12233/) (2018)
- [Attribution in Scale and Space](https://mlsanity.com/paper/2004.03383/) (2020)
- [Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution](https://mlsanity.com/paper/1802.06259/) (2018)
- [Universal Adversarial Perturbations: A Survey](https://mlsanity.com/paper/2005.08087/) (2020)
- [Computationally Efficient Measures of Internal Neuron Importance](https://mlsanity.com/paper/1807.09946/) (2018)
- [Understanding Integrated Gradients with SmoothTaylor for Deep Neural Network Attribution](https://mlsanity.com/paper/2004.10484/) (2020)
- [Symmetry-Preserving Paths in Integrated Gradients](https://mlsanity.com/paper/2103.13533/) (2021)
- [A New Method to Visualize Deep Neural Networks](https://mlsanity.com/paper/1603.02518/) (2016)
- [Explainability Tools Enabling Deep Learning in Future In-Situ Real-Time Planetary Explorations](https://mlsanity.com/paper/2201.05775/) (2022)

## Built on by (most cited first, 30 of 64)

- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons](https://mlsanity.com/paper/2308.13198/) (2023)
- [Sequential Integrated Gradients: a simple but effective method for explaining language models](https://mlsanity.com/paper/2305.15853/) (2023)
- [Knowledge Mechanisms in Large Language Models: A Survey and Perspective](https://mlsanity.com/paper/2407.15017/) (2024)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Neuron-Level Knowledge Attribution in Large Language Models](https://mlsanity.com/paper/2312.12141/) (2023)
- [Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda](https://mlsanity.com/paper/2404.09554/) (2024)
- [Manifold Integrated Gradients: Riemannian Geometry for Feature Attribution](https://mlsanity.com/paper/2405.09800/) (2024)
- [Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing](https://mlsanity.com/paper/2410.07054/) (2024)
- [Global Counterfactual Directions](https://mlsanity.com/paper/2404.12488/) (2024)
- [Deletion and Insertion Tests in Regression Models](https://mlsanity.com/paper/2205.12423/) (2022)
- [Stability Guarantees for Feature Attributions with Multiplicative Smoothing](https://mlsanity.com/paper/2307.05902/) (2023)
- [Knowledge Localization: Mission Not Accomplished? Enter Query Localization!](https://mlsanity.com/paper/2405.14117/) (2024)
- [FINER: Enhancing State-of-the-art Classifiers with Feature Attribution to Facilitate Security Analysis](https://mlsanity.com/paper/2308.05362/) (2023)
- [medXGAN: Visual Explanations for Medical Classifiers through a Generative Latent Space](https://mlsanity.com/paper/2204.05376/) (2022)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training](https://mlsanity.com/paper/2502.10927/) (2025)
- [Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactions](https://mlsanity.com/paper/2508.05430/) (2025)
- [Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency](https://mlsanity.com/paper/2603.17102/) (2026)
- [Non-Uniform Interpolation in Integrated Gradients for Low-Latency Explainable-AI](https://mlsanity.com/paper/2302.11107/) (2023)
- [Four Axiomatic Characterizations of the Integrated Gradients Attribution Method](https://mlsanity.com/paper/2306.13753/) (2023)
- [Comprehensive Attribution: Inherently Explainable Vision Model with Feature Detector](https://mlsanity.com/paper/2407.19308/) (2024)
- [The Gaussian Discriminant Variational Autoencoder (GdVAE): A Self-Explainable Model with Counterfactual Explanations](https://mlsanity.com/paper/2409.12952/) (2024)
- [Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models](https://mlsanity.com/paper/2502.20408/) (2025)
- [A Survey of Explainable AI in Deep Visual Modeling: Methods and Metrics](https://mlsanity.com/paper/2301.13445/) (2023)
- [Probabilistic Lipschitzness and the Stable Rank for Comparing Explanation Models](https://mlsanity.com/paper/2402.18863/) (2024)
- [Efficient and Accurate Explanation Estimation with Distribution Compression](https://mlsanity.com/paper/2406.18334/) (2024)

## Abstract (arXiv)

As deep learning (DL) efficacy grows, concerns for poor model explainability grow also. Attribution methods address the issue of explainability by quantifying the importance of an input feature for a model prediction. Among various methods, Integrated Gradients (IG) sets itself apart by claiming other methods failed to satisfy desirable axioms, while IG and methods like it uniquely satisfy said axioms. This paper comments on fundamental aspects of IG and its applications/extensions: 1) We identify key differences between IG function spaces and the supporting literature's function spaces which problematize previous claims of IG uniqueness. We show that with the introduction of an additional axiom, textitnon-decreasing positivity, the uniqueness claims can be established. 2) We address the question of input sensitivity by identifying function classes where IG is/is not Lipschitz in the attributed input. 3) We show that axioms for single-baseline methods have analogous properties for methods with probability distribution baselines. 4) We introduce a computationally efficient method of identifying internal neurons that contribute to specified regions of an IG attribution map. Finally, we present experimental results validating this method.