# From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance arXiv: https://arxiv.org/abs/2505.20229 ยท HTML: https://mlsanity.com/paper/2505.20229/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Interpreting vision and multimodal models, one of 13 directions in Mechanistic interpretability. - Builds on 41 of its 55 references that are in the map. - First posted on arXiv 26 May 2025. ## Placed in - Mechanistic interpretability / [Interpreting vision and multimodal models](https://mlsanity.com/mechinterp/interpreting-vision-and-multimodal-models/) ## Builds on (most cited first, 30 of 41) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016) - [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023) - [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022) - [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023) - [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021) - [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017) - [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023) - [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022) - [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023) - [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023) - [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021) - [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024) - [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017) - [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024) - [Data Filtering Networks](https://mlsanity.com/paper/2309.17425/) (2023) - [BCN20000: Dermoscopic Lesions in the Wild](https://mlsanity.com/paper/1908.02288/) (2019) - [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023) - [Interpreting CLIP's Image Representation via Text-Based Decomposition](https://mlsanity.com/paper/2310.05916/) (2023) - [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022) - [A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Context](https://mlsanity.com/paper/2008.07360/) (2020) - [From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation](https://mlsanity.com/paper/2206.03208/) (2022) - [Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)](https://mlsanity.com/paper/2402.10376/) (2024) - [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023) - [MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training](https://mlsanity.com/paper/2311.17049/) (2023) - [Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery](https://mlsanity.com/paper/2407.14499/) (2024) - [Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models](https://mlsanity.com/paper/2504.02821/) (2025) - [Sparse autoencoders reveal selective remapping of visual concepts during adaptation](https://mlsanity.com/paper/2412.05276/) (2024) - [Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment](https://mlsanity.com/paper/2502.03714/) (2025) - [Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNet](https://mlsanity.com/paper/2212.04871/) (2022) ## Built on by (most cited first, 6 of 6) - [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026) - [Model Science: getting serious about verification, explanation and control of AI systems](https://mlsanity.com/paper/2508.20040/) (2025) - [ELROND: Exploring and decomposing intrinsic capabilities of diffusion models](https://mlsanity.com/paper/2602.10216/) (2026) - [From Attribution to Action: A Human-Centered Application of Activation Steering](https://mlsanity.com/paper/2604.11467/) (2026) - [Towards Robustness against Typographic Attack with Training-free Concept Localization](https://mlsanity.com/paper/2607.02494/) (2026) - [Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models](https://mlsanity.com/paper/2610.14190/) (2026) ## Abstract (arXiv) Transformer-based CLIP models are widely used for text-image probing and feature extraction, making it relevant to understand the internal mechanisms behind their predictions. While recent works show that Sparse Autoencoders (SAEs) yield interpretable latent components, they focus on what these encode and miss how they drive predictions. We introduce a scalable framework that reveals what latent components activate for, how they align with expected semantics, and how important they are to predictions. To achieve this, we adapt attribution patching for instance-wise component attributions in CLIP and highlight key faithfulness limitations of the widely used Logit Lens technique. By combining attributions with semantic alignment scores, we can automatically uncover reliance on components that encode semantically unexpected or spurious concepts. Applied across multiple CLIP variants, our method uncovers hundreds of surprising components linked to polysemous words, compound nouns, visual typography and dataset artifacts. While text embeddings remain prone to semantic ambiguity, they are more robust to spurious correlations compared to linear classifiers trained on image embeddings. A case study on skin lesion detection highlights how such classifiers can amplify hidden shortcuts, underscoring the need for holistic, mechanistic interpretability. We provide code at https://github.com/maxdreyer/attributing-clip.