mlsanityData to 7 Oct 2026

Captum: A unified and generic model interpretability library for PyTorch

arXiv 2009.07896 ↗First posted 16 Sep 2020cs.LG cs.AI stat.MLcited by 542

Not on the map. Why?

Abstract (arXiv)

In this paper we introduce a novel, unified, open-source model interpretability library for PyTorch [12]. The library contains generic implementations of a number of gradient and perturbation-based attribution algorithms, also known as feature, neuron and layer importance algorithms, as well as a set of evaluation metrics for these algorithms. It can be used for both classification and non-classification models including graph-structured models built on Neural Networks (NN). In this paper we give a high-level overview of supported attribution algorithms and show how to perform memory-efficient and scalable computations. We emphasize that the three main characteristics of the library are multimodality, extensibility and ease of use. Multimodality supports different modality of inputs such as image, text, audio or video. Extensibility allows adding new algorithms and features. The library is also designed for easy understanding and use. Besides, we also introduce an interactive visualization tool called Captum Insights that is built on top of Captum library and allows sample-based model debugging and visualization using feature importance metrics.

Key facts. Cited by 542 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 52; 2022: 89; 2023: 99; 2024: 132; 2025: 95; 2026: 75.

542 papers.

Sort:

By area:

  1. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  2. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  3. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  4. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  5. 2021Do Concept Bottleneck Models Learn as Intended?cited by 114
  6. 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
  7. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  8. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 88
  9. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  10. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  11. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  12. 2021Improving Deep Learning Interpretability by Saliency Guided Trainingcited by 68
38 more
  1. 2022Measuring the Mixing of Contextual Information in the Transformercited by 63
  2. 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
  3. 2023Learning to Maximize Mutual Information for Dynamic Feature Selectioncited by 54
  4. 2023On Early Detection of Hallucinations in Factual Question Answeringcited by 54
  5. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 53
  6. 2022Evaluation of post-hoc interpretability methods in time-series classificationcited by 53
  7. 2023Sequential Integrated Gradients: a simple but effective method for explaining language modelscited by 52
  8. 2025PyG 2.0: Scalable Learning on Real World Graphscited by 49
  9. 2022Learning to Estimate Shapley Values with Vision Transformerscited by 44
  10. 2023Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistencycited by 42
  11. 2024shapiq: Shapley Interactions for Machine Learningcited by 42
  12. 2022Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanationscited by 40
  13. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  14. 2023SwiFT: Swin 4D fMRI Transformercited by 36
  15. 2022Modular and Parameter-Efficient Multimodal Fusion with Promptingcited by 35
  16. 2024Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Mediacited by 35
  17. 2022Fairness Reprogrammingcited by 33
  18. 2023Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Mapscited by 33
  19. 2023Using Captum to Explain Generative Language Modelscited by 33
  20. 2021UNIREX: A Unified Learning Framework for Language Model Rationale Extractioncited by 32
  21. 2022The Harvard USPTO Patent Dataset: A Large-Scale, Well-Structured, and Multi-Purpose Corpus of Patent Applicationscited by 32
  22. 2021Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processingcited by 30
  23. 2023AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulationcited by 30
  24. 2021Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behaviorcited by 29
  25. 2022On Explaining Multimodal Hateful Meme Detection Modelscited by 29
  26. 2022SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretabilitycited by 28
  27. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  28. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  29. 2021Temporal Dependencies in Feature Importance for Time Series Predictionscited by 27
  30. 2023Conditional Mutual Information for Disentangled Representations in Reinforcement Learningcited by 27
  31. 2021Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensingcited by 26
  32. 2022ferret: a Framework for Benchmarking Explainers on Transformerscited by 26
  33. 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspacescited by 25
  34. 2022Xplique: A Deep Learning Explainability Toolboxcited by 24
  35. 2021Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexicacited by 23
  36. 2024Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigationcited by 22
  37. 2021Robust Models Are More Interpretable Because Attributions Look Normalcited by 21
  38. 2022Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentationcited by 21

50 most cited of 542 All 542 (JSON)