mlsanityData to 14 Oct 2026

Explainability of Vision Transformers: A Comprehensive Review and New Perspectives

arXiv 2311.06786 ↗First posted 12 Nov 2023cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Transformers have had a significant impact on natural language processing and have recently demonstrated their potential in computer vision. They have shown promising results over convolution neural networks in fundamental computer vision tasks. However, the scientific community has not fully grasped the inner workings of vision transformers, nor the basis for their decision-making, which underscores the importance of explainability methods. Understanding how these models arrive at their decisions not only improves their performance but also builds trust in AI systems. This study explores different explainability methods proposed for visual transformers and presents a taxonomy for organizing them according to their motivations, structures, and application scenarios. In addition, it provides a comprehensive review of evaluation criteria that can be used for comparing explanation results, as well as explainability tools and frameworks. Finally, the paper highlights essential but unexplored aspects that can enhance the explainability of visual transformers, and promising research directions are suggested for future investment.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 62 of its 84 references that are in the map. First posted on arXiv 12 Nov 2023.

Builds on

62 of 84 references.

References · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  11. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  12. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
38 more
  1. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  2. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  3. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  4. 2019Attention is not Explanationcited by 929
  5. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  6. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  7. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  8. 2020Quantifying Attention Flow in Transformerscited by 775
  9. 2021Going deeper with Image Transformerscited by 764
  10. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  11. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  12. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  13. 2017Interpretation of Neural Networks is Fragilecited by 444
  14. 2021Transformer Trackingcited by 412
  15. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
  16. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
  17. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  18. 2023Explainability for Large Language Models: A Surveycited by 333
  19. 2016Top-down Neural Attention by Excitation Backpropcited by 321
  20. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 320
  21. 2019On the (In)fidelity and Sensitivity for Explanationscited by 310
  22. 2019On the Relationship between Self-Attention and Convolutional Layerscited by 300
  23. 2023Generating Images with Multimodal Language Modelscited by 260
  24. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 188
  25. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  26. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  27. 2016Grad-CAM: Why did you say that?cited by 161
  28. 2021All Tokens Matter: Token Labeling for Training Better Vision Transformerscited by 144
  29. 2021Patch Slimming for Efficient Vision Transformerscited by 133
  30. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  31. 2021A Survey of Visual Transformerscited by 117
  32. 2021Scalable Vision Transformers with Hierarchical Poolingcited by 84
  33. 2022What do Vision Transformers Learn? A Visual Explorationcited by 76
  34. 2020On quantitative aspects of model interpretabilitycited by 71
  35. 2023X-Pruner: eXplainable Pruning for Vision Transformerscited by 68
  36. 2019Understanding Neural Networks via Feature Visualization: A surveycited by 65
  37. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  38. 2021Convolutional Dynamic Alignment Networks for Interpretable Classificationscited by 54

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 2.

12 papers.

  1. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  2. 2025Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysiscited by 12
  3. 2025Evaluating the Explainability of Vision Transformers in Medical Imagingcited by 4
  4. 2025A Hybrid Fully Convolutional CNN-Transformer Model for Inherently Interpretable Disease Detection from Retinal Fundus Imagescited by 3
  5. 2025EDIT: Enhancing Vision Transformers by Mitigating Attention Sink through an Encoder-Decoder Architecturecited by 2
  6. 2025X-SiT: Inherently Interpretable Surface Vision Transformers for Dementia Diagnosiscited by 1
  7. 2025An Explainable Transformer Model for Alzheimer's Disease Detection Using Retinal Imagingcited by 1
  8. 2026Geometric Capacity of Transformers: A Tropical Geometry Perspectivecited by 1
  9. 2024A Multi-Modal Explainability Approach for Human-Aware Robots in Multi-Party Conversationcited by 0
  10. 2024ViTmiX: Vision Transformer Explainability Augmented by Mixed Visualization Methodscited by 0
  11. 2025Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablationscited by 0
  12. 2026DINO-QPM: Adapting Visual Foundation Models for Globally Interpretable Image Classificationcited by 0