# The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision

arXiv: https://arxiv.org/abs/2406.03662 · HTML: https://mlsanity.com/paper/2406.03662/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Builds on (most cited first, 7 of 7)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Linear Algebraic Structure of Word Senses, with Applications to Polysemy](https://mlsanity.com/paper/1601.03764/) (2016)
- [Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors](https://mlsanity.com/paper/2103.15949/) (2021)
- [An Interpretability Illusion for BERT](https://mlsanity.com/paper/2104.07143/) (2021)

## Built on by (most cited first, 30 of 37)

- [SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders](https://mlsanity.com/paper/2501.18052/) (2025)
- [Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models](https://mlsanity.com/paper/2502.12892/) (2025)
- [Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment](https://mlsanity.com/paper/2502.03714/) (2025)
- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [Steering CLIP's vision transformer with sparse autoencoders](https://mlsanity.com/paper/2504.08729/) (2025)
- [Interpreting the linear structure of vision-language model embedding spaces](https://mlsanity.com/paper/2504.11695/) (2025)
- [Do Sparse Autoencoders Capture Concept Manifolds?](https://mlsanity.com/paper/2604.28119/) (2026)
- [An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation](https://mlsanity.com/paper/2410.03334/) (2024)
- [Mechanistic?](https://mlsanity.com/paper/2410.09087/) (2024)
- [Block-Recurrent Dynamics in Vision Transformers](https://mlsanity.com/paper/2512.19941/) (2025)
- [Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders](https://mlsanity.com/paper/2411.13117/) (2024)
- [TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation](https://mlsanity.com/paper/2503.07050/) (2025)
- [Learning biologically relevant features in a pathology foundation model using sparse autoencoders](https://mlsanity.com/paper/2407.10785/) (2024)
- [SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoders](https://mlsanity.com/paper/2509.21379/) (2025)
- [Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings](https://mlsanity.com/paper/2602.06218/) (2026)
- [Evaluating SAE interpretability without explanations](https://mlsanity.com/paper/2507.08473/) (2025)
- [Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds](https://mlsanity.com/paper/2606.25234/) (2026)
- [Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models](https://mlsanity.com/paper/2412.16247/) (2024)
- [Learning Concept Bottleneck Models from Mechanistic Explanations](https://mlsanity.com/paper/2603.07343/) (2026)
- [Group Crosscoders for Mechanistic Analysis of Symmetry](https://mlsanity.com/paper/2410.24184/) (2024)
- [A Geometric Unification of Concept Learning with Concept Cones](https://mlsanity.com/paper/2512.07355/) (2025)
- [Insights on Galaxy Evolution from Interpretable Sparse Feature Networks](https://mlsanity.com/paper/2501.00089/) (2024)
- [Measuring the (Un)Faithfulness of Concept-Based Explanations](https://mlsanity.com/paper/2504.10833/) (2025)
- [Decoding Vision Transformers: the Diffusion Steering Lens](https://mlsanity.com/paper/2504.13763/) (2025)
- [Sparse Autoencoders for Sequential Recommendation Models: Interpretation and Flexible Control](https://mlsanity.com/paper/2507.12202/) (2025)
- [Language Models Can Explain Visual Features via Steering](https://mlsanity.com/paper/2603.22593/) (2026)
- [On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders](https://mlsanity.com/paper/2605.31518/) (2026)
- [Uncovering Branch specialization in InceptionV1 using k sparse autoencoders](https://mlsanity.com/paper/2504.11489/) (2025)
- [Explaining How Visual, Textual and Multimodal Encoders Share Concepts](https://mlsanity.com/paper/2507.18512/) (2025)
- [Backdoor Directions in Vision Transformers](https://mlsanity.com/paper/2603.10806/) (2026)

## Abstract (arXiv)

Recent work on sparse autoencoders (SAEs) has shown promise in extracting interpretable features from neural networks and addressing challenges with polysemantic neurons caused by superposition. In this paper, we apply SAEs to the early vision layers of InceptionV1, a well-studied convolutional neural network, with a focus on curve detectors. Our results demonstrate that SAEs can uncover new interpretable features not apparent from examining individual neurons, including additional curve detectors that fill in previous gaps. We also find that SAEs can decompose some polysemantic neurons into more monosemantic constituent features. These findings suggest SAEs are a valuable tool for understanding InceptionV1, and convolutional neural networks more generally.