mlsanityData to 7 Oct 2026

Grad-SAM: Explaining Transformers via Gradient Self-Attention Maps

arXiv 2204.11073 ↗First posted 23 Apr 2022cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

Transformer-based language models significantly advanced the state-of-the-art in many linguistic tasks. As this revolution continues, the ability to explain model predictions has become a major area of interest for the NLP community. In this work, we present Gradient Self-Attention Maps (Grad-SAM) - a novel gradient-based method that analyzes self-attention units and identifies the input elements that explain the model's prediction the best. Extensive evaluations on various benchmarks show that Grad-SAM obtains significant improvements over state-of-the-art alternatives.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 23 Apr 2022.

Builds on

36 of 56 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  8. 2018Deep contextualized word representationscited by 2,285
  9. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  10. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  11. 2019Attention is not Explanationcited by 916
  12. 2020A Primer in BERTology: What we know about how BERT workscited by 807
24 more
  1. 2020Quantifying Attention Flow in Transformerscited by 759
  2. 2019Attention is not not Explanationcited by 629
  3. 2019Is Attention Interpretable?cited by 387
  4. 2015Semi-supervised Sequence Learningcited by 198
  5. 2019BERT for Coreference Resolution: Baselines and Analysiscited by 125
  6. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  7. 2020Learning to Faithfully Rationalize by Constructioncited by 113
  8. 2019Learning to Deceive with Attention-Based Explanationscited by 111
  9. 2019Do Attention Heads in BERT Track Syntactic Dependencies?cited by 87
  10. 2019AllenNLP Interpret: A Framework for Explaining Predictions of NLP Modelscited by 74
  11. 2019U-CAM: Visual Explanation using Uncertainty based Class Activation Mapscited by 36
  12. 2020Explainable Recommendations via Attentive Multi-Persona Collaborative Filteringcited by 15
  13. 2021Self-Supervised Document Similarity Ranking via Contextualized Language Models and Hierarchical Inferencecited by 13
  14. 2019Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embeddingcited by 12
  15. 2019Explanation vs Attention: A Two-Player Game to Obtain Attention for VQAcited by 12
  16. 2016Bayesian Neural Word Embeddingcited by 11
  17. 2020Neural Attentive Multiview Machinescited by 11
  18. 2020Bayesian Hierarchical Words Representation Learningcited by 10
  19. 2020Attentive Item2Vec: Neural Attentive User Representationscited by 9
  20. 2020RecoBERT: A Catalog Language Model for Text-Based Recommendationscited by 8
  21. 2021GAM: Explainable Visual Similarity and Classification via Gradient Activation Mapscited by 8
  22. 2021Cold Item Integration in Deep Hybrid Recommenders via Tunable Stochastic Gatescited by 8
  23. 2021SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoningcited by 7
  24. 2021Normal vs. Adversarial: Salience-based Analysis of Adversarial Samples for Relation Extractioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 10; 2024: 8; 2025: 9; 2026: 14.

42 papers.

Sort:

By area:

  1. 2023Explainability for Large Language Models: A Surveycited by 331
  2. 2023From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuningcited by 56
  3. 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 47
  4. 2023B-cos Alignment for Inherently Interpretable CNNs and Vision Transformerscited by 31
  5. 2024Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agendacited by 22
  6. 2024Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformercited by 19
  7. 2023Visual Explanations via Iterated Integrated Attributionscited by 18
  8. 2023Explainability of Vision Transformers: A Comprehensive Review and New Perspectivescited by 12
  9. 2023Deep Integrated Explanationscited by 9
  10. 2025The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer trainingcited by 9
  11. 2022Interpreting BERT-based Text Similarity via Activation and Saliency Mapscited by 8
  12. 2023Holistically Explainable Vision Transformerscited by 8
30 more
  1. 2023Interpretability-Aware Vision Transformercited by 8
  2. 2024Better Explain Transformers by Illuminating Important Informationcited by 8
  3. 2025Evaluating the Effectiveness of XAI Techniques for Encoder-Based Language Modelscited by 7
  4. 2023Weakly Supervised Intracranial Hemorrhage Segmentation using Head-Wise Gradient-Infused Self-Attention Maps from a Swin Transformer in Categorical Learningcited by 6
  5. 2024ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scalecited by 5
  6. 2025FinTRec: Transformer Based Unified Contextual Ads Targeting and Personalization for Financial Applicationscited by 4
  7. 2023Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Modelscited by 3
  8. 2025Generalized Attention Flow: Feature Attribution for Transformer Models via Maximum Flowcited by 3
  9. 2024Reversed Attention: On The Gradient Descent Of Attention Layers In GPTcited by 2
  10. 2025Task-Related Token Compression in Multimodal Large Language Models from an Explainability Perspectivecited by 2
  11. 2025Guided Perturbation Sensitivity (GPS): Detecting Adversarial Text via Embedding Stability and Word Importancecited by 2
  12. 2026Concept-Guided Fine-Tuning: Steering ViTs away from Spurious Correlations to Improve Robustnesscited by 2
  13. 2026Causal Attribution via Activation Patchingcited by 2
  14. 2024Enhancing Conditional Image Generation with Explainable Latent Space Manipulationcited by 1
  15. 2025Pruning the Paradox: How CLIP's Most Informative Heads Enhance Performance While Amplifying Biascited by 1
  16. 2025Contrast-CAT: Contrasting Activations for Enhanced Interpretability in Transformer-based Text Classifierscited by 1
  17. 2026From Attribution to Abstention: Training-Free Attention-Based Auditing for Clinical Summarizationcited by 1
  18. 2024Automated Trustworthiness Oracle Generation for Machine Learning Text Classifierscited by 0
  19. 2025Integrating attention into explanation frameworks for language and vision transformerscited by 0
  20. 2026Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Informationcited by 0
  21. 2026Learning to Explain: Supervised Token Attribution from Transformer Attention Patternscited by 0
  22. 2026TimeSliver : Symbolic-Linear Decomposition for Explainable Time Series Classificationcited by 0
  23. 2026Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inferencecited by 0
  24. 2026OmniTrace: A Unified Framework for Generation-Time Attribution in Omni-Modal LLMscited by 0
  25. 2026Benchmarking Multi-Modal Graph-based Social Media Popularity Predictioncited by 0
  26. 2026Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformerscited by 0
  27. 2026HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Datacited by 0
  28. 2026GARLIC: Graph Attention-based Relational Learning of Multivariate Time Series in Intensive Carecited by 0
  29. 2026Faithful Faithfulness Evaluations: Challenges & Pitfalls Learned from a Breast MRI Case Studycited by 0
  30. 2026ASAP: Visual Analytics for Identifying and Analyzing Image Patterns in AI-generated Imagescited by 0