# Discretized Integrated Gradients for Explaining Language Models

arXiv: https://arxiv.org/abs/2108.13654 · HTML: https://mlsanity.com/paper/2108.13654/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 31 Aug 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs](https://mlsanity.com/paper/1801.05453/) (2018)
- [Hierarchical interpretations for neural network predictions](https://mlsanity.com/paper/1806.05337/) (2018)
- [Investigating Saturation Effects in Integrated Gradients](https://mlsanity.com/paper/2010.12697/) (2020)
- [Generalized Integrated Gradients: A practical method for explaining diverse ensembles](https://mlsanity.com/paper/1909.01869/) (2019)

## Built on by (most cited first, 30 of 46)

- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://mlsanity.com/paper/2302.13942/) (2023)
- [Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons](https://mlsanity.com/paper/2308.13198/) (2023)
- [Sequential Integrated Gradients: a simple but effective method for explaining language models](https://mlsanity.com/paper/2305.15853/) (2023)
- [SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features](https://mlsanity.com/paper/2206.07023/) (2022)
- [UNIREX: A Unified Learning Framework for Language Model Rationale Extraction](https://mlsanity.com/paper/2112.08802/) (2021)
- [Backward Lens: Projecting Language Model Gradients into the Vocabulary Space](https://mlsanity.com/paper/2402.12865/) (2024)
- [FaiRR: Faithful and Robust Deductive Reasoning over Natural Language](https://mlsanity.com/paper/2203.10261/) (2022)
- [ferret: a Framework for Benchmarking Explainers on Transformers](https://mlsanity.com/paper/2208.01575/) (2022)
- [AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution](https://mlsanity.com/paper/2411.15102/) (2024)
- [Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations](https://mlsanity.com/paper/2504.12691/) (2025)
- [Knowledge Localization: Mission Not Accomplished? Enter Query Localization!](https://mlsanity.com/paper/2405.14117/) (2024)
- [Hard to Explain: On the Computational Hardness of In-Distribution Model Interpretation](https://mlsanity.com/paper/2408.03915/) (2024)
- [FRAME: Evaluating Rationale-Label Consistency Metrics for Free-Text Rationales](https://mlsanity.com/paper/2207.00779/) (2022)
- [An Attribution Method for Siamese Encoders](https://mlsanity.com/paper/2310.05703/) (2023)
- [Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Features](https://mlsanity.com/paper/2309.07733/) (2023)
- [ER-Test: Evaluating Explanation Regularization Methods for Language Models](https://mlsanity.com/paper/2205.12542/) (2022)
- [Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictions](https://mlsanity.com/paper/2305.10614/) (2023)
- [Reasoning-Grounded Natural Language Explanations for Language Models](https://mlsanity.com/paper/2503.11248/) (2025)
- [Time Interpret: a Unified Model Interpretability Library for Time Series](https://mlsanity.com/paper/2306.02968/) (2023)
- [Jacobian Scopes: token-level causal attributions in LLMs](https://mlsanity.com/paper/2601.16407/) (2026)
- [Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insights](https://mlsanity.com/paper/2208.06096/) (2022)
- [Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Models](https://mlsanity.com/paper/2408.11252/) (2024)
- [Explaining Caption-Image Interactions in CLIP Models with Second-Order Attributions](https://mlsanity.com/paper/2408.14153/) (2024)
- [One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models](https://mlsanity.com/paper/2411.17401/) (2024)
- [Identification of Knowledge Neurons in Protein Language Models](https://mlsanity.com/paper/2312.10770/) (2023)
- ["Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing](https://mlsanity.com/paper/2403.18976/) (2024)
- [Uniform Discretized Integrated Gradients: An effective attribution based method for explaining large language models](https://mlsanity.com/paper/2412.03886/) (2024)
- [Hessian-Enhanced Token Attribution (HETA): Interpreting Autoregressive LLMs](https://mlsanity.com/paper/2604.13258/) (2026)
- [Generalizability Analysis of Graph-based Trajectory Predictor with Vectorized Representation](https://mlsanity.com/paper/2208.03578/) (2022)

## Abstract (arXiv)

As a prominent attribution-based explanation algorithm, Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. It measures feature importance by averaging the model's output gradient interpolated along a straight-line path in the input data space. However, such straight-line interpolated points are not representative of text data due to the inherent discreteness of the word embedding space. This questions the faithfulness of the gradients computed at the interpolated points and consequently, the quality of the generated explanations. Here we propose Discretized Integrated Gradients (DIG), which allows effective attribution along non-linear interpolation paths. We develop two interpolation strategies for the discrete word embedding space that generates interpolation points that lie close to actual words in the embedding space, yielding more faithful gradient computation. We demonstrate the effectiveness of DIG over IG through experimental and human evaluations on multiple sentiment classification datasets. We provide the source code of DIG to encourage reproducible research.