mlsanityData to 7 Oct 2026

Quantifying Attention Flow in Transformers

arXiv 2005.00928 ↗First posted 2 May 2020cs.LG cs.AI cs.CLcited by 759

Not on the map. Why?

Cited by 60 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In the Transformer model, "self-attention" combines information from attended embeddings into the representation of the focal embedding in the next layer. Thus, across layers of the Transformer, information originating from different tokens gets increasingly mixed. This makes attention weights unreliable as explanations probes. In this paper, we consider the problem of quantifying this flow of information through self-attention. We propose two methods for approximating the attention to input tokens given attention weights, attention rollout and attention flow, as post hoc methods when we use attention weights as the relative relevance of the input tokens. We show that these methods give complementary views on the flow of information, and compared to raw attention, both yield higher correlations with importance scores of input tokens obtained using an ablation method and input gradients.

Key facts. Cited by 759 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 54; 2022: 95; 2023: 102; 2024: 137; 2025: 156; 2026: 215.

759 papers.

Sort:

By area:

  1. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  2. 2021Transformers in Vision: A Surveycited by 1,009
  3. 2022A Generalist Agentcited by 802
  4. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  5. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  6. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  7. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
  8. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  9. 2021Vision Transformers are Robust Learnerscited by 208
  10. 2021Anticipative Video Transformercited by 197
  11. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  12. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 179
38 more
  1. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  2. 2021Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformercited by 175
  3. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  4. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  5. 2021Conformer: Local Features Coupling Global Representations for Visual Recognitioncited by 164
  6. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  7. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  8. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  9. 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 131
  10. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  11. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  12. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 123
  13. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  14. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  15. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  16. 2021VidTr: Video Transformer Without Convolutionscited by 112
  17. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102
  18. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  19. 2021Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understandingcited by 97
  20. 2024ContextCite: Attributing Model Generation to Contextcited by 93
  21. 2021Inpainting Transformer for Anomaly Detectioncited by 91
  22. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  23. 2024The Hidden Attention of Mamba Modelscited by 88
  24. 2022A graph-transformer for whole slide image classificationcited by 84
  25. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  26. 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agentscited by 81
  27. 2021Causal Attention for Unbiased Visual Recognitioncited by 80
  28. 2021TransMix: Attend to Mix for Vision Transformerscited by 79
  29. 2022Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identificationcited by 78
  30. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  31. 2024DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Modelscited by 74
  32. 2021Probing Inter-modality: Visual Parsing with Self-Attention for Vision-Language Pre-trainingcited by 67
  33. 2023X-Pruner: eXplainable Pruning for Vision Transformerscited by 66
  34. 2022Delving into Masked Autoencoders for Multi-Label Thorax Disease Classificationcited by 65
  35. 2022Video Transformers: A Surveycited by 63
  36. 2022Measuring the Mixing of Contextual Information in the Transformercited by 63
  37. 2022ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognitioncited by 63
  38. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 63

50 most cited of 759 All 759 (JSON)