mlsanityData to 7 Oct 2026

Using Captum to Explain Generative Language Models

arXiv 2312.05491 ↗First posted 9 Dec 2023cs.CL cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

Captum is a comprehensive library for model explainability in PyTorch, offering a range of methods from the interpretability literature to enhance users' understanding of PyTorch models. In this paper, we introduce new features in Captum that are specifically designed to analyze the behavior of generative language models. We provide an overview of the available functionalities and example applications of their potential for understanding learned associations within generative language models.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 9 Dec 2023.

Builds on

11 of 20 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  5. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  6. 2018Sanity Checks for Saliency Mapscited by 1,149
  7. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  8. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  9. 2020OpenAttack: An Open-source Textual Adversarial Attack Toolkitcited by 84
  10. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  11. 2023Data Augmentation for Neural NLPcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 8; 2026: 5.

33 papers.

Sort:

By area:

  1. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  2. 2024Understanding the Dark Side of LLMs' Intrinsic Self-Correctioncited by 42
  3. 2024Backward Lens: Projecting Language Model Gradients into the Vocabulary Spacecited by 30
  4. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  5. 2024LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Modelscited by 16
  6. 2024CLEAR: Can Language Models Really Understand Causal Graphs?cited by 14
  7. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patternscited by 13
  8. 2024Learning to Understand: Identifying Interactions via the M"obius Transformcited by 12
  9. 2024SyntaxShap: Syntax-aware Explainability Method for Text Generationcited by 11
  10. 2024Multi-Level Explanations for Generative Language Modelscited by 11
  11. 2025Explainability in Practice: A Survey of Explainable NLP Across Various Domainscited by 11
  12. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 11
21 more
  1. 2024Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanationscited by 8
  2. 2024Interactive Prompt Debugging with Sequence Saliencecited by 6
  3. 2024JoPA:Explaining Large Language Model's Generation via Joint Prompt Attributioncited by 6
  4. 2024ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMscited by 6
  5. 2024SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generationcited by 6
  6. 2025Towards Achieving Concept Completeness for Textual Concept Bottleneck Modelscited by 6
  7. 2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 5
  8. 2024DUE: Dynamic Uncertainty-Aware Explanation Supervision via 3D Imputationcited by 4
  9. 2024Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Modelscited by 3
  10. 2025SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controllingcited by 3
  11. 2025Understanding the Dilemma of Unlearning for Large Language Modelscited by 3
  12. 2024Selective Explanationscited by 2
  13. 2024Local Explanations and Self-Explanations for Assessing Faithfulness in black-box LLMscited by 2
  14. 2025EvalxNLP: A Framework for Benchmarking Post-Hoc Explainability Methods on NLP Modelscited by 2
  15. 2026Interpreting and Controlling Model Behavior via Constitutions for Atomic Concept Editscited by 2
  16. 2024Effective Large Language Model Debugging with Best-first Tree Searchcited by 1
  17. 2025Document Attribution: Examining Citation Relationships using Large Language Modelscited by 1
  18. 2026LLM attribution analysis across different fine-tuning strategies and model scales for automated code compliancecited by 1
  19. 2026Identifying Good and Bad Neurons for Task-Level Controllable LLMscited by 0
  20. 2026The Attribution Contract for Generative Language Modelscited by 0
  21. 2026Temporally-Resolved Token Attribution Reveals the Generation Dynamics of Diffusion Language Modelscited by 0