# Dynamic Accumulated Attention Map for Interpreting Evolution of Decision-Making in Vision Transformer

arXiv: https://arxiv.org/abs/2503.14640 · HTML: https://mlsanity.com/paper/2503.14640/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [Explaining NonLinear Classification Decisions with Deep Taylor Decomposition](https://mlsanity.com/paper/1512.02479/) (2015)
- [Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs](https://mlsanity.com/paper/2008.02312/) (2020)
- [Efficient Self-supervised Vision Transformers for Representation Learning](https://mlsanity.com/paper/2106.09785/) (2021)
- [All Tokens Matter: Token Labeling for Training Better Vision Transformers](https://mlsanity.com/paper/2104.10858/) (2021)
- [MedViT: A Robust Vision Transformer for Generalized Medical Image Classification](https://mlsanity.com/paper/2302.09462/) (2023)
- [RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2301.10222/) (2023)
- [GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection](https://mlsanity.com/paper/2104.14528/) (2021)
- [Visual Explanations via Iterated Integrated Attributions](https://mlsanity.com/paper/2310.18585/) (2023)
- [ReViT: Enhancing Vision Transformers Feature Diversity with Attention Residual Connections](https://mlsanity.com/paper/2402.11301/) (2024)

## Built on by (most cited first, 12 of 12)

- [Feature Complementation Architecture for Visual Place Recognition](https://mlsanity.com/paper/2506.12401/) (2025)
- [Second-order Gaussian directional derivative representations for image high-resolution corner detection](https://mlsanity.com/paper/2601.08182/) (2026)
- [Deep learning-based neurodevelopmental assessment in preterm infants](https://mlsanity.com/paper/2601.11944/) (2026)
- [Adaptive receptive field-based spatial-frequency feature reconstruction network for fine-grained few-shot image classification](https://mlsanity.com/paper/2604.16936/) (2026)
- [GloResNet: A lightweight 3D CNN with global topological features for preterm brain injury prediction](https://mlsanity.com/paper/2606.02498/) (2026)
- [Spatial frequency information fusion network for few-shot learning](https://mlsanity.com/paper/2506.18364/) (2025)
- [Video Understanding by Design: How Datasets Shape Video Models](https://mlsanity.com/paper/2509.09151/) (2025)
- [TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching](https://mlsanity.com/paper/2601.19739/) (2026)
- [Uncertainty-DTW for Sequences and Visual Tokens](https://mlsanity.com/paper/2605.25110/) (2026)
- [Structured Hyperedge Adaptation for Parameter-Efficient Fine-Tuning of Vision Transformers](https://mlsanity.com/paper/2606.22383/) (2026)
- [HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitives](https://mlsanity.com/paper/2609.01282/) (2026)
- [Phase-Aware Spatial-Frequency Fusion for Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/2609.03829/) (2026)

## Abstract (arXiv)

Various Vision Transformer (ViT) models have been widely used for image recognition tasks. However, existing visual explanation methods can not display the attention flow hidden inside the inner structure of ViT models, which explains how the final attention regions are formed inside a ViT for its decision-making. In this paper, a novel visual explanation approach, Dynamic Accumulated Attention Map (DAAM), is proposed to provide a tool that can visualize, for the first time, the attention flow from the top to the bottom through ViT networks. To this end, a novel decomposition module is proposed to construct and store the spatial feature information by unlocking the [class] token generated by the self-attention module of each ViT block. The module can also obtain the channel importance coefficients by decomposing the classification score for supervised ViT models. Because of the lack of classification score in self-supervised ViT models, we propose dimension-wise importance weights to compute the channel importance coefficients. Such spatial features are linearly combined with the corresponding channel importance coefficients, forming the attention map for each block. The dynamic attention flow is revealed by block-wisely accumulating each attention map. The contribution of this work focuses on visualizing the evolution dynamic of the decision-making attention for any intermediate block inside a ViT model by proposing a novel decomposition module and dimension-wise importance weights. The quantitative and qualitative analysis consistently validate the effectiveness and superior capacity of the proposed DAAM for not only interpreting ViT models with the fully-connected layers as the classifier but also self-supervised ViT models. The code is available at https://github.com/ly9802/DynamicAccumulatedAttentionMap.