mlsanityData to 14 Oct 2026

Beyond Semantics: Disentangling Information Scope in Sparse Autoencoders for CLIP

arXiv 2604.05724 ↗First posted 7 Apr 2026cs.CVcited by 1

On the map: Mechanistic interpretability → Interpreting vision and multimodal models

Abstract (arXiv)

Sparse Autoencoders (SAEs) have emerged as a powerful tool for interpreting the internal representations of CLIP vision encoders, yet existing analyses largely focus on the semantic meaning of individual features. We introduce information scope as a complementary dimension of interpretability that characterizes how broadly an SAE feature aggregates visual evidence, ranging from localized, patch-specific cues to global, image-level signals. We observe that some SAE features respond consistently across spatial perturbations, while others shift unpredictably with minor input changes, indicating a fundamental distinction in their underlying scope. To quantify this, we propose the Contextual Dependency Score (CDS), which separates positionally stable local scope features from positionally variant global scope features. Our experiments show that features of different information scopes exert systematically different influences on CLIP's predictions and confidence. These findings establish information scope as a critical new axis for understanding CLIP representations and provide a deeper diagnostic view of SAE-derived features.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Interpreting vision and multimodal models, one of 13 directions in Mechanistic interpretability. Builds on 23 of its 37 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

23 of 37 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Causal Interpretation of Sparse Autoencoder Features in Vision31 Aug
  2. 2025Probing the Representational Power of Sparse Autoencoders in Vision Models15 Aug
  3. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models3 Apr
  4. 2025Interpreting CLIP with Hierarchical Sparse Autoencoders27 Feb
  5. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment6 Feb
  6. 2024Sparse autoencoders reveal selective remapping of visual concepts during adaptation6 Dec
  7. 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Models22 Nov

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  6. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  7. 2022Toy Models of Superpositioncited by 971
  8. 2023Vision Transformers Need Registerscited by 871
  9. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  10. 2024Scaling and evaluating sparse autoencoderscited by 553
  11. 2022DeiT III: Revenge of the ViTcited by 419
  12. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
4 more
  1. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 61
  2. 2025Vision Transformers Don't Need Trained Registerscited by 41
  3. 2025Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrievalcited by 8
  4. 2025Pruning the Paradox: How CLIP's Most Informative Heads Enhance Performance While Amplifying Biascited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Geometric Similarity in VLM Low-Level Vision Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Sparse Autoencoders enable Robust and Interpretable Fine-tuning of CLIP models
  2. 2025Interpretable and Testable Vision Features via Sparse Autoencoders
  3. 2026Sparse CLIP: Co-Optimizing Interpretability and Performance in Contrastive Learning
  4. 2026From Weights to Concepts: Data-Free Interpretability of CLIP via Singular Vector Decomposition
  5. 2026Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
  6. 2025Beyond Interpretability: When, Why, and How Sparse Autoencoders Enable Label-Free Visual Steering
  7. 2025Steering CLIP's vision transformer with sparse autoencoders
  8. 2026Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?