mlsanityData to 7 Oct 2026

The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision

arXiv 2406.03662 ↗First posted 6 Jun 2024cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

Recent work on sparse autoencoders (SAEs) has shown promise in extracting interpretable features from neural networks and addressing challenges with polysemantic neurons caused by superposition. In this paper, we apply SAEs to the early vision layers of InceptionV1, a well-studied convolutional neural network, with a focus on curve detectors. Our results demonstrate that SAEs can uncover new interpretable features not apparent from examining individual neurons, including additional curve detectors that fill in previous gaps. We also find that SAEs can decompose some polysemantic neurons into more monosemantic constituent features. These findings suggest SAEs are a valuable tool for understanding InceptionV1, and convolutional neural networks more generally.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 7 of its 18 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

7 of 18 references.

References · 7

  1. 2014Going Deeper with Convolutionscited by 4,231
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  3. 2022Toy Models of Superpositioncited by 928
  4. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  5. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  6. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 106
  7. 2021An Interpretability Illusion for BERTcited by 94

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 19; 2026: 11.

37 papers.

Sort:

By area:

  1. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  2. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  3. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  4. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 43
  5. 2025Steering CLIP's vision transformer with sparse autoencoderscited by 33
  6. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  7. 2024Mechanistic?cited by 25
  8. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
  9. 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 23
  10. 2024Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoderscited by 17
  11. 2025TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generationcited by 16
  12. 2025Block-Recurrent Dynamics in Vision Transformerscited by 16
25 more
  1. 2024Learning biologically relevant features in a pathology foundation model using sparse autoencoderscited by 14
  2. 2026Cross-Modal Redundancy and the Geometry of Vision-Language Embeddingscited by 9
  3. 2025SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoderscited by 8
  4. 2025Evaluating SAE interpretability without explanationscited by 5
  5. 2024Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation modelscited by 4
  6. 2026Learning Concept Bottleneck Models from Mechanistic Explanationscited by 4
  7. 2024Group Crosscoders for Mechanistic Analysis of Symmetrycited by 3
  8. 2025A Geometric Unification of Concept Learning with Concept Conescited by 3
  9. 2026Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifoldscited by 3
  10. 2024Insights on Galaxy Evolution from Interpretable Sparse Feature Networkscited by 2
  11. 2025Sparse Autoencoders for Sequential Recommendation Models: Interpretation and Flexible Controlcited by 2
  12. 2026On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoderscited by 2
  13. 2025Measuring the (Un)Faithfulness of Concept-Based Explanationscited by 1
  14. 2025Uncovering Branch specialization in InceptionV1 using k sparse autoencoderscited by 1
  15. 2025Decoding Vision Transformers: the Diffusion Steering Lenscited by 1
  16. 2025Explaining How Visual, Textual and Multimodal Encoders Share Conceptscited by 1
  17. 2026Backdoor Directions in Vision Transformerscited by 1
  18. 2026Language Models Can Explain Visual Features via Steeringcited by 1
  19. 2026A Unifying Framework for Concept-Based Representational Similaritycited by 1
  20. 2025Naturally Computed Scale Invariance in the Residual Stream of ResNet18cited by 0
  21. 2025Relevance-driven Input Dropout: an Explanation-guided Regularization Techniquecited by 0
  22. 2025Concept-SAE: A Controllable and Invertible Concept Interface for Sparse Autoencoderscited by 0
  23. 2025Probing the Probes: Methods and Metrics for Concept Alignmentcited by 0
  24. 2026Do Multilingual LLMs have specialized language heads?cited by 0
  25. 2026What Cohort INRs Encode and Where to Freeze Themcited by 0