mlsanityData to 14 Oct 2026

[Re] Improving Interpretation Faithfulness for Vision Transformers

arXiv 2509.14846 ↗First posted 18 Sep 2025cs.CV cs.AIcited by 1

On the map: Mechanistic interpretability → Interpreting vision and multimodal models

Abstract (arXiv)

This work aims to reproduce the results of Faithful Vision Transformers (FViTs) proposed by arXiv:2311.17983 alongside interpretability methods for Vision Transformers from arXiv:2012.09838 and Xu (2022) et al. We investigate claims made by arXiv:2311.17983, namely that the usage of Diffusion Denoised Smoothing (DDS) improves interpretability robustness to (1) attacks in a segmentation task and (2) perturbation and attacks in a classification task. We also extend the original study by investigating the authors' claims that adding DDS to any interpretability method can improve its robustness under attack. This is tested on baseline methods and the recently proposed Attribution Rollout method. In addition, we measure the computational costs and environmental impact of obtaining an FViT through DDS. Our results broadly agree with the original study's findings, although minor discrepancies were found and discussed.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Interpreting vision and multimodal models, one of 13 directions in Mechanistic interpretability. Builds on 23 of its 26 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

23 of 26 references.

References · 23

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  9. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  10. 2020Pre-Trained Image Processing Transformercited by 893
  11. 2020Quantifying Attention Flow in Transformerscited by 775
  12. 2022On Distillation of Guided Diffusion Modelscited by 725
11 more
  1. 2017Renyi Differential Privacycited by 650
  2. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  3. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  4. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  5. 2016Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layerscited by 199
  6. 2018Certified Adversarial Robustness with Additive Noisecited by 180
  7. 2023Diffusion Models With Learned Adaptive Noisecited by 51
  8. 2019Issues with post-hoc counterfactual explanations: a discussioncited by 30
  9. 2023Improving Interpretation Faithfulness for Vision Transformerscited by 12
  10. 2021Certifiably Robust Interpretation via Renyi Differential Privacycited by 10
  11. 2024Machine Learning Robustness: A Primercited by 8

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Benchmarking and Mitigating Sycophancy in Medical Vision Language Modelscited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2023VISION DIFFMASK: Faithful Interpretation of Vision Transformers with Differentiable Patch Masking
  2. 2026Taming Outlier Tokens in Diffusion Transformers
  3. 2023Interpretability-Aware Vision Transformer
  4. 2026Sparse but not Simpler: A Multi-Level Interpretability Analysis of Vision Transformers
  5. 2025Robust Representation Consistency Model via Contrastive Denoising
  6. 2026A Mechanistic Analysis of Adversarial Fine-tuning of Vision Transformers
  7. 2025Beyond Classification: Evaluating Diffusion Denoised Smoothing for Security-Utility Trade off
  8. 2024ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers