mlsanityData to 14 Oct 2026

Sparse autoencoders reveal selective remapping of visual concepts during adaptation

arXiv 2412.05276 ↗First posted 6 Dec 2024cs.CV cs.LGcited by 68

On the map: Mechanistic interpretability → Interpreting vision and multimodal models

Abstract (arXiv)

Adapting foundation models for specific purposes has become a standard approach to build machine learning systems for downstream applications. Yet, it is an open question which mechanisms take place during adaptation. Here we develop a new Sparse Autoencoder (SAE) for the CLIP vision transformer, named PatchSAE, to extract interpretable concepts at granular levels (e.g., shape, color, or semantics of an object) and their patch-wise spatial attributions. We explore how these concepts influence the model output in downstream image classification tasks and investigate how recent state-of-the-art prompt-based adaptation techniques change the association of model inputs to these concepts. While activations of concepts slightly change between adapted and non-adapted models, we find that the majority of gains on common adaptation tasks can be explained with the existing concepts already present in the non-adapted foundation model. This work provides a concrete framework to train and use SAEs for Vision Transformers and provides insights into explaining adaptation mechanisms.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). Placed in Interpreting vision and multimodal models, one of 13 directions in Mechanistic interpretability. Builds on 18 of its 37 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

18 of 37 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  9. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  10. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  11. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  12. 2022MaPLe: Multi-modal Prompt Learningcited by 687
6 more
  1. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  2. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 274
  3. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  4. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 179
  5. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 108
  6. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 27; 2026: 41.

68 papers.

Sort:

By area:

  1. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 71
  2. 2025Steering CLIP's vision transformer with sparse autoencoderscited by 38
  3. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  4. 2025Interpretable and Testable Vision Features via Sparse Autoencoderscited by 25
  5. 2025Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Videocited by 21
  6. 2025From superposition to sparse codes: interpretable representations in neural networkscited by 16
  7. 2025On the Limits of Sparse Autoencoders: A Theoretical Framework and Reweighted Remedycited by 14
  8. 2026DNA: Uncovering Universal Latent Forgery Knowledgecited by 14
  9. 2025VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Setcited by 13
  10. 2025CytoSAE: Interpretable Cell Embeddings for Hematologycited by 8
  11. 2025Adversarial Attacks Leverage Interference Between Features in Superpositioncited by 8
  12. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
38 more
  1. 2025From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliancecited by 6
  2. 2025Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactionscited by 6
  3. 2025Causal Interpretation of Sparse Autoencoder Features in Visioncited by 5
  4. 2025Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Modelscited by 5
  5. 2025ConceptScope: Characterizing Dataset Bias via Disentangled Visual Conceptscited by 5
  6. 2025SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucinationcited by 5
  7. 2026Probing Cross-modal Information Hubs in Audio-Visual LLMscited by 5
  8. 2025Symbolic Rule Extraction from Attention-Guided Sparse Representations in Vision Transformerscited by 3
  9. 2026Latent Anomaly Knowledge Excavation: Unveiling Sparse Sensitive Neurons in Vision-Language Modelscited by 3
  10. 2026Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformerscited by 3
  11. 2025Measuring the (Un)Faithfulness of Concept-Based Explanationscited by 2
  12. 2025Beyond Interpretability: When, Why, and How Sparse Autoencoders Enable Label-Free Visual Steeringcited by 2
  13. 2025Mammo-SAE: Interpreting Breast Cancer Concept Learning with Sparse Autoencoderscited by 2
  14. 2026Sparse CLIP: Co-Optimizing Interpretability and Performance in Contrastive Learningcited by 2
  15. 2026Language Models Can Explain Visual Features via Steeringcited by 2
  16. 2026Improving Sparse Autoencoder with Dynamic Attentioncited by 2
  17. 2026Sparse Autoencoders enable Robust and Interpretable Fine-tuning of CLIP modelscited by 2
  18. 2026Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoderscited by 2
  19. 2026Extraction and Analysis of Multimodal Concepts in Vision Language Models through Sparse Autoencoderscited by 2
  20. 2026Evaluating the Interpretability of Sparse Autoencoders with Concept Annotationscited by 2
  21. 2025VITA: Zero-Shot Value Functions via Test-Time Adaptation of Vision-Language Modelscited by 1
  22. 2025Explaining How Visual, Textual and Multimodal Encoders Share Conceptscited by 1
  23. 2025ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Modelscited by 1
  24. 2025FaCT: Faithful Concept Traces for Explaining Neural Network Decisionscited by 1
  25. 2025SALVE: Sparse Autoencoder-Latent Vector Editing for Mechanistic Control of Neural Networkscited by 1
  26. 2025LouvreSAE: Sparse Autoencoders for Interpretable and Controllable Style Transfercited by 1
  27. 2026LUCID-SAE: Learning Unified Vision-Language Sparse Codes for Interpretable Concept Discoverycited by 1
  28. 2026MonoLoss: A Training Objective for Interpretable Monosemantic Representationscited by 1
  29. 2026VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Modelscited by 1
  30. 2026SCoCCA: Multi-modal Sparse Concept Decomposition via Canonical Correlation Analysiscited by 1
  31. 2026Beyond Semantics: Disentangling Information Scope in Sparse Autoencoders for CLIPcited by 1
  32. 2026From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Modelscited by 1
  33. 2026GeoSAE: Geometric Prior-Guided Layer-Wise Sparse Autoencoder Annotation of Brain MRI Foundation Modelscited by 1
  34. 2026A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principlecited by 1
  35. 2026Interpretability Transfer from Language to Vision via Sparse Autoencoderscited by 1
  36. 2026Cascaded Sparse Autoencoders Learn Multi-Level Visual Concepts in Multimodal LLMscited by 1
  37. 2026When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalitiescited by 1
  38. 2025Concept-SAE: A Controllable and Invertible Concept Interface for Sparse Autoencoderscited by 0

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Probing the Representational Power of Sparse Autoencoders in Vision Models
  2. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment
  3. 2026Look But Don't Touch with Sparse Autoencoders for Unlearning in Diffusion Models
  4. 2024Interpreting Attention Layer Outputs with Sparse Autoencoders
  5. 2025Interpreting CLIP with Hierarchical Sparse Autoencoders
  6. 2026Supervised sparse auto-encoders for interpretable and compositional representations
  7. 2024Analyzing (In)Abilities of SAEs via Formal Languages
  8. 2025Interpreting the linear structure of vision-language model embedding spaces