# How Does Attention Work in Vision Transformers? A Visual Analytics Attempt

arXiv: https://arxiv.org/abs/2303.13731 · HTML: https://mlsanity.com/paper/2303.13731/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Self-Attention Attribution: Interpreting Information Interactions Inside Transformer](https://mlsanity.com/paper/2004.11207/) (2020)
- [Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://mlsanity.com/paper/1801.06889/) (2018)
- [Identifying and Controlling Important Neurons in Neural Machine Translation](https://mlsanity.com/paper/1811.01157/) (2018)
- [Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2005.07310/) (2020)
- [Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models](https://mlsanity.com/paper/1804.09299/) (2018)
- [VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers](https://mlsanity.com/paper/2203.17247/) (2022)
- [Towards Better Analysis of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1604.07043/) (2016)
- [Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models](https://mlsanity.com/paper/2009.07053/) (2020)
- [Visual Analytics for Explainable Deep Learning](https://mlsanity.com/paper/1804.02527/) (2018)
- [SANVis: Visual Analytics for Understanding Self-Attention Networks](https://mlsanity.com/paper/1909.09595/) (2019)
- [VisQA: X-raying Vision and Language Reasoning in Transformers](https://mlsanity.com/paper/2104.00926/) (2021)
- [GNNLens: A Visual Analytics Approach for Prediction Error Diagnosis of Graph Neural Networks](https://mlsanity.com/paper/2011.11048/) (2020)
- [T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP](https://mlsanity.com/paper/2108.13587/) (2021)
- [A Visual Analytics System for Improving Attention-based Traffic Forecasting Models](https://mlsanity.com/paper/2208.04350/) (2022)

## Built on by (most cited first, 30 of 30)

- [Foundation Models Meet Visualizations: Challenges and Opportunities](https://mlsanity.com/paper/2310.05771/) (2023)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2411.17150/) (2024)
- [GMAR: Gradient-Driven Multi-Head Attention Rollout for Vision Transformer Interpretability](https://mlsanity.com/paper/2504.19414/) (2025)
- [Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentation](https://mlsanity.com/paper/2408.04619/) (2024)
- [Stop Misusing t-SNE and UMAP for Visual Analytics](https://mlsanity.com/paper/2506.08725/) (2025)
- [Charts-of-Thought: Enhancing LLM Visualization Literacy Through Structured Data Extraction](https://mlsanity.com/paper/2508.04842/) (2025)
- [Improving Interpretability and Robustness for the Detection of AI-Generated Images](https://mlsanity.com/paper/2406.15035/) (2024)
- [Visual Analytics for Efficient Image Exploration and User-Guided Image Captioning](https://mlsanity.com/paper/2311.01016/) (2023)
- [Visualization for Trust in Machine Learning Revisited: The State of the Field in 2023](https://mlsanity.com/paper/2403.12005/) (2024)
- [Illuminating LLM Coding Agents: Visual Analytics for Deeper Understanding and Enhancement](https://mlsanity.com/paper/2508.12555/) (2025)
- [Model Science: getting serious about verification, explanation and control of AI systems](https://mlsanity.com/paper/2508.20040/) (2025)
- [When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs](https://mlsanity.com/paper/2509.24258/) (2025)
- [VSSFlow: Unifying Video-conditioned Sound and Speech Generation via Joint Learning](https://mlsanity.com/paper/2509.24773/) (2025)
- [Vision Transformer attention alignment with human visual perception in aesthetic object evaluation](https://mlsanity.com/paper/2507.17616/) (2025)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [Eyes on Target: Gaze-Aware Object Detection in Egocentric Video](https://mlsanity.com/paper/2511.01237/) (2025)
- [KAN-Mixers: a new deep learning architecture for image classification](https://mlsanity.com/paper/2503.08939/) (2025)
- [Multimedia and Visual Analytics in the Agentic Era](https://mlsanity.com/paper/2504.06138/) (2025)
- [VisMoDAl: Visual Analytics for Evaluating and Improving Corruption Robustness of Vision-Language Models](https://mlsanity.com/paper/2509.14571/) (2025)
- [An Evaluation of Hybrid Annotation Workflows on High-Ambiguity Spatiotemporal Video Footage](https://mlsanity.com/paper/2510.21798/) (2025)
- [Weight-Decay Turns Transformer Loss Landscapes Villani: Functional-Analytic Foundations for Optimization and Generalization](https://mlsanity.com/paper/2605.06599/) (2026)
- [Understanding Transformer-based Vision Models through Inversion](https://mlsanity.com/paper/2412.06534/) (2024)
- [Visual Attention Exploration in Vision-Based Mamba Models](https://mlsanity.com/paper/2502.20764/) (2025)
- [Relational inductive biases on attention mechanisms](https://mlsanity.com/paper/2507.04117/) (2025)
- [SHMoAReg: Spark Deformable Image Registration via Spatial Heterogeneous Mixture of Experts and Attention Heads](https://mlsanity.com/paper/2509.20073/) (2025)
- [ViT-Explainer: An Interactive Walkthrough of the Vision Transformer Pipeline](https://mlsanity.com/paper/2604.02182/) (2026)
- [RepTran: Search-Based Repair of Transformer Models](https://mlsanity.com/paper/2607.11193/) (2026)
- [Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models](https://mlsanity.com/paper/2609.22097/) (2026)
- [Data-Adaptive Mahalanobis Metric Learning for Cross-Head Attention in Transformers](https://mlsanity.com/paper/2610.13261/) (2026)

## Abstract (arXiv)

Vision transformer (ViT) expands the success of transformer models from sequential data to images. The model decomposes an image into many smaller patches and arranges them into a sequence. Multi-head self-attentions are then applied to the sequence to learn the attention between patches. Despite many successful interpretations of transformers on sequential data, little effort has been devoted to the interpretation of ViTs, and many questions remain unanswered. For example, among the numerous attention heads, which one is more important? How strong are individual patches attending to their spatial neighbors in different heads? What attention patterns have individual heads learned? In this work, we answer these questions through a visual analytics approach. Specifically, we first identify what heads are more important in ViTs by introducing multiple pruning-based metrics. Then, we profile the spatial distribution of attention strengths between patches inside individual heads, as well as the trend of attention strengths across attention layers. Third, using an autoencoder-based learning solution, we summarize all possible attention patterns that individual heads could learn. Examining the attention strengths and patterns of the important heads, we answer why they are important. Through concrete case studies with experienced deep learning experts on multiple ViTs, we validate the effectiveness of our solution that deepens the understanding of ViTs from head importance, head attention strength, and head attention pattern.