mlsanityData to 7 Oct 2026

ViT-CX: Causal Explanation of Vision Transformers

arXiv 2211.03064 ↗First posted 6 Nov 2022cs.CV cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

Despite the popularity of Vision Transformers (ViTs) and eXplainable AI (XAI), only a few explanation methods have been designed specially for ViTs thus far. They mostly use attention weights of the [CLS] token on patch embeddings and often produce unsatisfactory saliency maps. This paper proposes a novel method for explaining ViTs called ViT-CX. It is based on patch embeddings, rather than attentions paid to them, and their causal impacts on the model output. Other characteristics of ViTs such as causal overdetermination are also considered in the design of ViT-CX. The empirical results show that ViT-CX produces more meaningful saliency maps and does a better job revealing all important evidence for the predictions than previous methods. The explanation generated by ViT-CX also shows significantly better faithfulness to the model. The codes and appendix are available at https://github.com/vaynexie/CausalX-ViT.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 6 Nov 2022.

Builds on

42 of 54 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  12. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
30 more
  1. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  2. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  4. 2018Sanity Checks for Saliency Mapscited by 1,149
  5. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  6. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  7. 2019Attention is not Explanationcited by 916
  8. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  9. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  10. 2020Quantifying Attention Flow in Transformerscited by 759
  11. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  12. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  13. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  14. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  15. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  16. 2021Intriguing Properties of Vision Transformerscited by 446
  17. 2019Is Attention Interpretable?cited by 387
  18. 2016Top-down Neural Attention by Excitation Backpropcited by 318
  19. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
  20. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  21. 2016Understanding Neural Networks through Representation Erasurecited by 223
  22. 2021Vision Transformers are Robust Learnerscited by 208
  23. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  24. 2019Learning to Deceive with Attention-Based Explanationscited by 111
  25. 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
  26. 2021Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understandingcited by 97
  27. 2020There and Back Again: Revisiting Backpropagation Saliency Methodscited by 60
  28. 2022Learning to Estimate Shapley Values with Vision Transformerscited by 44
  29. 2020Explaining Convolutional Neural Networks through Attribution-Based Input Sampling and Block-Wise Feature Aggregationcited by 13
  30. 2021Contrastive Counterfactual Visual Explanations With Overdeterminationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 7; 2026: 14.

27 papers.

Sort:

By area:

  1. 2023The Co-12 Recipe for Evaluating Interpretable Part-Prototype Image Classifierscited by 15
  2. 2023Explainability of Vision Transformers: A Comprehensive Review and New Perspectivescited by 12
  3. 2025GMAR: Gradient-Driven Multi-Head Attention Rollout for Vision Transformer Interpretabilitycited by 9
  4. 2025Register and [CLS] tokens yield a decoupling of local and global features in large ViTscited by 8
  5. 2025Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIPcited by 7
  6. 2024Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Modelscited by 6
  7. 2025Comprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformerscited by 6
  8. 2024LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributionscited by 5
  9. 2024FairViT: Fair Vision Transformer via Adaptive Maskingcited by 3
  10. 2026Causal Attribution via Activation Patchingcited by 2
  11. 2026Debunking Grad-ECLIP: A Comprehensive Study on Its Incorrectness and Fundamental Principles for Model Interpretationcited by 2
  12. 2025Multi-Head Explainer: A General Framework to Improve Explainability in CNNs and Transformerscited by 1
15 more
  1. 2025Two-stage Vision Transformers and Hard Masking offer Robust Object Representationscited by 1
  2. 2026FB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglementcited by 1
  3. 2026PhaseWin: An Efficient Search Algorithm for Faithful Visual Attributioncited by 1
  4. 2024FilterViT and DropoutViTcited by 0
  5. 2025Do Protein Transformers Have Biological Intelligence?cited by 0
  6. 2026Learn to Rank: Visual Attribution by Learning Importance Rankingcited by 0
  7. 2026ATTN-FIQA: Interpretable Attention-based Face Image Quality Assessment with Vision Transformerscited by 0
  8. 2026Transformer Interpretability from Perspective of Attention and Gradientcited by 0
  9. 2026The Neglected Baseline in Model Interpretationcited by 0
  10. 2026Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structurescited by 0
  11. 2026A Good Initialization is All You Need for Faithful Visual Attributioncited by 0
  12. 2026Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformerscited by 0
  13. 2026Fine-grained CLIP fine-tuning with self-annotated region alignmentcited by 0
  14. 2026Explaining AI-Image Detection: What the Heatmap Actually Showscited by 0
  15. 2026Unraveling the Real Working Mechanism and Inherent Flaws of GAE: A Method for Interpreting Transformer Processes from an Economic Perspectivecited by 0