# Decomposing and Interpreting Image Representations via Text in ViTs Beyond CLIP

arXiv: https://arxiv.org/abs/2406.01583 · HTML: https://mlsanity.com/paper/2406.01583/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [MaxViT: Multi-Axis Vision Transformer](https://mlsanity.com/paper/2204.01697/) (2022)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Revisiting Model Stitching to Compare Neural Representations](https://mlsanity.com/paper/2106.07682/) (2021)
- [Interpreting CLIP's Image Representation via Text-Based Decomposition](https://mlsanity.com/paper/2310.05916/) (2023)
- [CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks](https://mlsanity.com/paper/2204.10965/) (2022)
- [Analyzing Vision Transformers for Image Classification in Class Embedding Space](https://mlsanity.com/paper/2310.18969/) (2023)
- [Decomposing and Editing Predictions by Modeling Model Computation](https://mlsanity.com/paper/2404.11534/) (2024)
- [Towards Improved Input Masking for Convolutional Neural Networks](https://mlsanity.com/paper/2211.14646/) (2022)

## Built on by (most cited first, 20 of 20)

- [Interpreting CLIP with Hierarchical Sparse Autoencoders](https://mlsanity.com/paper/2502.20578/) (2025)
- [MoH: Multi-Head Attention as Mixture-of-Head Attention](https://mlsanity.com/paper/2410.11842/) (2024)
- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations](https://mlsanity.com/paper/2511.22697/) (2025)
- [Probing the Representational Power of Sparse Autoencoders in Vision Models](https://mlsanity.com/paper/2508.11277/) (2025)
- [Understanding Multi-View Transformers](https://mlsanity.com/paper/2510.24907/) (2025)
- [ResiDual Transformer Alignment with Spectral Decomposition](https://mlsanity.com/paper/2411.00246/) (2024)
- [Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretation](https://mlsanity.com/paper/2506.13831/) (2025)
- [Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactions](https://mlsanity.com/paper/2508.05430/) (2025)
- [Sparse Fine-Tuning of Transformers for Generative Tasks](https://mlsanity.com/paper/2507.10855/) (2025)
- [Exploring and Leveraging Class Vectors for Classifier Editing](https://mlsanity.com/paper/2510.11268/) (2025)
- [Zero-Shot Textual Explanations via Translating Decision-Critical Features](https://mlsanity.com/paper/2512.07245/) (2025)
- [Diffusion-CAM: Faithful Visual Explanations for dMLLMs](https://mlsanity.com/paper/2604.11005/) (2026)
- [Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Models](https://mlsanity.com/paper/2605.06640/) (2026)
- [TRANSPORTER: Transferring Visual Semantics from VLM Manifolds](https://mlsanity.com/paper/2511.18359/) (2025)
- [Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generation](https://mlsanity.com/paper/2602.15650/) (2026)
- [Zero-Shot Faithful Textual Explanations via Directional-Derivative Influence on Predictions](https://mlsanity.com/paper/2605.16877/) (2026)
- [IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers](https://mlsanity.com/paper/2608.05122/) (2026)
- [Grad-CAM for Vision Transformers: A Systematic Taxonomy and Audit of Methodological Ambiguity in Explainable AI](https://mlsanity.com/paper/2608.05258/) (2026)
- [Metonymic Circuits for Abstract Concept Grounding in Vision Transformers](https://mlsanity.com/paper/2610.06928/) (2026)

## Abstract (arXiv)

Recent work has explored how individual components of the CLIP-ViT model contribute to the final representation by leveraging the shared image-text representation space of CLIP. These components, such as attention heads and MLPs, have been shown to capture distinct image features like shape, color or texture. However, understanding the role of these components in arbitrary vision transformers (ViTs) is challenging. To this end, we introduce a general framework which can identify the roles of various components in ViTs beyond CLIP. Specifically, we (a) automate the decomposition of the final representation into contributions from different model components, and (b) linearly map these contributions to CLIP space to interpret them via text. Additionally, we introduce a novel scoring function to rank components by their importance with respect to specific features. Applying our framework to various ViT variants (e.g. DeiT, DINO, DINOv2, Swin, MaxViT), we gain insights into the roles of different components concerning particular image features. These insights facilitate applications such as image retrieval using text descriptions or reference images, visualizing token importance heatmaps, and mitigating spurious correlations. We release our code to reproduce the experiments at https://github.com/SriramB-98/vit-decompose