# Towards Evaluating Explanations of Vision Transformers for Medical Imaging

arXiv: https://arxiv.org/abs/2304.06133 · HTML: https://mlsanity.com/paper/2304.06133/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers](https://mlsanity.com/paper/2103.15679/) (2021)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [TransFER: Learning Relation-aware Facial Expression Representations with Transformers](https://mlsanity.com/paper/2108.11116/) (2021)
- [On quantitative aspects of model interpretability](https://mlsanity.com/paper/2007.07584/) (2020)
- [COVID-19 Infection Localization and Severity Grading from Chest X-ray Images](https://mlsanity.com/paper/2103.07985/) (2021)
- [Understanding Individual Decisions of CNNs via Contrastive Backpropagation](https://mlsanity.com/paper/1812.02100/) (2018)
- [Multi-view analysis of unregistered medical images using cross-view transformers](https://mlsanity.com/paper/2103.11390/) (2021)
- [Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation](https://mlsanity.com/paper/1908.04351/) (2019)
- [Vision Transformer using Low-level Chest X-ray Feature Corpus for COVID-19 Diagnosis and Severity Quantification](https://mlsanity.com/paper/2104.07235/) (2021)
- [Checklist for responsible deep learning modeling of medical images based on COVID-19 detection studies](https://mlsanity.com/paper/2012.08333/) (2020)
- [Bayesian Optimization using Pseudo-Points](https://mlsanity.com/paper/1910.05484/) (2019)
- [Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Images](https://mlsanity.com/paper/2206.02761/) (2022)

## Built on by (most cited first, 23 of 23)

- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video](https://mlsanity.com/paper/2504.19475/) (2025)
- [Medical Slice Transformer: Improved Diagnosis and Explainability on 3D Medical Images with DINOv2](https://mlsanity.com/paper/2411.15802/) (2024)
- [Explainability of Vision Transformers: A Comprehensive Review and New Perspectives](https://mlsanity.com/paper/2311.06786/) (2023)
- [Evaluating Visual Explanations of Attention Maps for Transformer-based Medical Imaging](https://mlsanity.com/paper/2503.09535/) (2025)
- [Aggregated Attributions for Explanatory Analysis of 3D Segmentation Models](https://mlsanity.com/paper/2407.16653/) (2024)
- [Evaluating the Explainability of Vision Transformers in Medical Imaging](https://mlsanity.com/paper/2510.12021/) (2025)
- [DAVE: Distribution-aware Attribution via ViT Gradient Decomposition](https://mlsanity.com/paper/2602.06613/) (2026)
- [Be Careful When Evaluating Explanations Regarding Ground Truth](https://mlsanity.com/paper/2311.04813/) (2023)
- [PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification](https://mlsanity.com/paper/2404.08915/) (2024)
- [Revisiting FunnyBirds evaluation framework for prototypical parts networks](https://mlsanity.com/paper/2408.11401/) (2024)
- [CNN-based explanation ensembling for dataset, representation and explanations evaluation](https://mlsanity.com/paper/2404.10387/) (2024)
- [X-SiT: Inherently Interpretable Surface Vision Transformers for Dementia Diagnosis](https://mlsanity.com/paper/2506.20267/) (2025)
- [DXAI: Explaining Classification by Image Decomposition](https://mlsanity.com/paper/2401.00320/) (2023)
- [Learning to Project for Cross-Task Knowledge Distillation](https://mlsanity.com/paper/2403.14494/) (2024)
- [DFE-IANet: A Method for Polyp Image Classification Based on Dual-domain Feature Extraction and Interaction Attention](https://mlsanity.com/paper/2407.20843/) (2024)
- [PiPViT: Patch-based Visual Interpretable Prototypes for Retinal Image Analysis](https://mlsanity.com/paper/2506.10669/) (2025)
- [Topological Conditioning for Mammography Models via a Stable Wavelet-Persistence Vectorization](https://mlsanity.com/paper/2512.10151/) (2025)
- [Uncertainty-Calibrated Explainable Artificial Intelligence for Fetal Ultrasound Plane Classification: A Systematic Review](https://mlsanity.com/paper/2601.00990/) (2026)
- [Med-SegLens: Latent-Level Model Diffing for Interpretable Medical Image Segmentation](https://mlsanity.com/paper/2602.10508/) (2026)
- [MedConcept: Unsupervised Concept Discovery for Interpretability in Medical VLMs](https://mlsanity.com/paper/2604.11868/) (2026)
- [FZ-VLM: A Two Stage Florence-Zephyr Vision Language Model Framework for Pulmonary Nodule Characterization and Clinical Decision Making](https://mlsanity.com/paper/2608.15004/) (2026)
- [Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation](https://mlsanity.com/paper/2610.12703/) (2026)

## Abstract (arXiv)

As deep learning models increasingly find applications in critical domains such as medical imaging, the need for transparent and trustworthy decision-making becomes paramount. Many explainability methods provide insights into how these models make predictions by attributing importance to input features. As Vision Transformer (ViT) becomes a promising alternative to convolutional neural networks for image classification, its interpretability remains an open research question. This paper investigates the performance of various interpretation methods on a ViT applied to classify chest X-ray images. We introduce the notion of evaluating faithfulness, sensitivity, and complexity of ViT explanations. The obtained results indicate that Layerwise relevance propagation for transformers outperforms Local interpretable model-agnostic explanations and Attention visualization, providing a more accurate and reliable representation of what a ViT has actually learned. Our findings provide insights into the applicability of ViT explanations in medical imaging and highlight the importance of using appropriate evaluation criteria for comparing them.