Disentangling Visual Transformers: Patch-level Interpretability for Image Classification
Not on the map. Why?
Abstract (arXiv)
Visual transformers have achieved remarkable performance in image classification tasks, but this performance gain has come at the cost of interpretability. One of the main obstacles to the interpretation of transformers is the self-attention mechanism, which mixes visual information across the whole image in a complex way. In this paper, we propose Hindered Transformer (HiT), a novel interpretable by design architecture inspired by visual transformers. Our proposed architecture rethinks the design of transformers to better disentangle patch influences at the classification stage. Ultimately, HiT can be interpreted as a linear combination of patch-level information. We show that the advantages of our approach in terms of explicability come with a reasonable trade-off in performance, making it an attractive alternative for applications where interpretability is paramount.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 57 of its 76 references that are in the map. First posted on arXiv 24 Feb 2025.
Builds on
57 of 76 references.
References · 57
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Visual Instruction Tuningcited by 7,792
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2016Layer Normalizationcited by 5,579
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
38 more
- 2015Learning Deep Features for Discriminative Localizationcited by 2,430
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
- 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
- 2017Deformable Convolutional Networkscited by 1,481
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2018Sanity Checks for Saliency Mapscited by 1,149
- 2020Concept Bottleneck Modelscited by 1,022
- 2019Attention is not Explanationcited by 916
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
- 2020Quantifying Attention Flow in Transformerscited by 759
- 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
- 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
- 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
- 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
- 2022DeiT III: Revenge of the ViTcited by 417
- 2019Is Attention Interpretable?cited by 387
- 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
- 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
- 2023Label-Free Concept Bottleneck Modelscited by 282
- 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
- 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
- 2022MetaFormer Baselines for Visioncited by 131
- 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
- 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
- 2022Diffusion Visual Counterfactual Explanationscited by 96
- 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
- 2021Scalable Vision Transformers with Hierarchical Poolingcited by 84
- 2022Diffusion Models for Counterfactual Explanationscited by 79
- 2019Interpretable Image Recognition with Hierarchical Prototypescited by 64
- 2022ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognitioncited by 63
- 2022Concept-level Debugging of Part-Prototype Networkscited by 52
- 2020This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognitioncited by 50
- 2023Adversarial Counterfactual Visual Explanationscited by 47
- 2023Learning Support and Trivial Prototypes for Interpretable Image Classificationcited by 38
- 2021A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Conceptscited by 33
- 2023B-cos Alignment for Inherently Interpretable CNNs and Vision Transformerscited by 31
- 2021CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliencycited by 28
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.