# ViT-CX: Causal Explanation of Vision Transformers

arXiv: https://arxiv.org/abs/2211.03064 · HTML: https://mlsanity.com/paper/2211.03064/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 54 references that are in the map.
- First posted on arXiv 6 Nov 2022.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)

## Built on by (most cited first, 28 of 28)

- [The Co-12 Recipe for Evaluating Interpretable Part-Prototype Image Classifiers](https://mlsanity.com/paper/2307.14517/) (2023)
- [Explainability of Vision Transformers: A Comprehensive Review and New Perspectives](https://mlsanity.com/paper/2311.06786/) (2023)
- [GMAR: Gradient-Driven Multi-Head Attention Rollout for Vision Transformer Interpretability](https://mlsanity.com/paper/2504.19414/) (2025)
- [Register and [CLS] tokens yield a decoupling of local and global features in large ViTs](https://mlsanity.com/paper/2505.05892/) (2025)
- [Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIP](https://mlsanity.com/paper/2502.18816/) (2025)
- [Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models](https://mlsanity.com/paper/2406.12649/) (2024)
- [Comprehensive Information Bottleneck for Unveiling Universal Attribution to Interpret Vision Transformers](https://mlsanity.com/paper/2507.04388/) (2025)
- [LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributions](https://mlsanity.com/paper/2411.16760/) (2024)
- [FairViT: Fair Vision Transformer via Adaptive Masking](https://mlsanity.com/paper/2407.14799/) (2024)
- [Causal Attribution via Activation Patching](https://mlsanity.com/paper/2603.13652/) (2026)
- [Debunking Grad-ECLIP: A Comprehensive Study on Its Incorrectness and Fundamental Principles for Model Interpretation](https://mlsanity.com/paper/2605.12952/) (2026)
- [Multi-Head Explainer: A General Framework to Improve Explainability in CNNs and Transformers](https://mlsanity.com/paper/2501.01311/) (2025)
- [Two-stage Vision Transformers and Hard Masking offer Robust Object Representations](https://mlsanity.com/paper/2506.08915/) (2025)
- [FB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglement](https://mlsanity.com/paper/2603.19608/) (2026)
- [PhaseWin: An Efficient Search Algorithm for Faithful Visual Attribution](https://mlsanity.com/paper/2606.18008/) (2026)
- [FilterViT and DropoutViT](https://mlsanity.com/paper/2410.22709/) (2024)
- [Do Protein Transformers Have Biological Intelligence?](https://mlsanity.com/paper/2506.06701/) (2025)
- [Learn to Rank: Visual Attribution by Learning Importance Ranking](https://mlsanity.com/paper/2604.05819/) (2026)
- [ATTN-FIQA: Interpretable Attention-based Face Image Quality Assessment with Vision Transformers](https://mlsanity.com/paper/2604.22841/) (2026)
- [Transformer Interpretability from Perspective of Attention and Gradient](https://mlsanity.com/paper/2605.11392/) (2026)
- [The Neglected Baseline in Model Interpretation](https://mlsanity.com/paper/2605.22417/) (2026)
- [Generic Interpretation Approach for Transformer Models Incorporating Heterogenous Attention Structures](https://mlsanity.com/paper/2605.27458/) (2026)
- [A Good Initialization is All You Need for Faithful Visual Attribution](https://mlsanity.com/paper/2607.06726/) (2026)
- [Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers](https://mlsanity.com/paper/2607.10365/) (2026)
- [Fine-grained CLIP fine-tuning with self-annotated region alignment](https://mlsanity.com/paper/2607.13661/) (2026)
- [Explaining AI-Image Detection: What the Heatmap Actually Shows](https://mlsanity.com/paper/2607.29581/) (2026)
- [Discovery and Spatial Characterisation of Multiple Shortcut Groups for Auditing Vision Model Bias](https://mlsanity.com/paper/2608.14051/) (2026)
- [Unraveling the Real Working Mechanism and Inherent Flaws of GAE: A Method for Interpreting Transformer Processes from an Economic Perspective](https://mlsanity.com/paper/2609.07213/) (2026)

## Abstract (arXiv)

Despite the popularity of Vision Transformers (ViTs) and eXplainable AI (XAI), only a few explanation methods have been designed specially for ViTs thus far. They mostly use attention weights of the [CLS] token on patch embeddings and often produce unsatisfactory saliency maps. This paper proposes a novel method for explaining ViTs called ViT-CX. It is based on patch embeddings, rather than attentions paid to them, and their causal impacts on the model output. Other characteristics of ViTs such as causal overdetermination are also considered in the design of ViT-CX. The empirical results show that ViT-CX produces more meaningful saliency maps and does a better job revealing all important evidence for the predictions than previous methods. The explanation generated by ViT-CX also shows significantly better faithfulness to the model. The codes and appendix are available at https://github.com/vaynexie/CausalX-ViT.