# Do Sparse Autoencoders Capture Concept Manifolds?

arXiv: https://arxiv.org/abs/2604.28119 · HTML: https://mlsanity.com/paper/2604.28119/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- Builds on 71 of its 123 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Builds on (most cited first, 30 of 71)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders](https://mlsanity.com/paper/2407.14435/) (2024)
- [AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders](https://mlsanity.com/paper/2501.17148/) (2025)
- [Not All Language Model Features Are One-Dimensionally Linear](https://mlsanity.com/paper/2405.14860/) (2024)
- [Open Problems in Mechanistic Interpretability](https://mlsanity.com/paper/2501.16496/) (2025)
- [Complexity of Linear Regions in Deep Networks](https://mlsanity.com/paper/1901.09021/) (2019)
- [Linear Algebraic Structure of Word Senses, with Applications to Polysemy](https://mlsanity.com/paper/1601.03764/) (2016)
- [Bounding and Counting Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1711.02114/) (2017)
- [A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders](https://mlsanity.com/paper/2409.14507/) (2024)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [Learning Multi-Level Features with Matryoshka Sparse Autoencoders](https://mlsanity.com/paper/2503.17547/) (2025)
- [BatchTopK Sparse Autoencoders](https://mlsanity.com/paper/2412.06410/) (2024)
- [Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding](https://mlsanity.com/paper/2007.10930/) (2020)
- [Are Sparse Autoencoders Useful? A Case Study in Sparse Probing](https://mlsanity.com/paper/2502.16681/) (2025)
- [The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence](https://mlsanity.com/paper/2502.17420/) (2025)
- [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023)
- [Deep Subspace Clustering Networks](https://mlsanity.com/paper/1709.02508/) (2017)
- [Sparse Autoencoders Trained on the Same Data Learn Different Features](https://mlsanity.com/paper/2501.16615/) (2025)
- [Representation Benefits of Deep Feedforward Networks](https://mlsanity.com/paper/1509.08101/) (2015)
- [On the Origins of Linear Representations in Large Language Models](https://mlsanity.com/paper/2403.03867/) (2024)
- [Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models](https://mlsanity.com/paper/2502.12892/) (2025)
- [Language Models Use Trigonometry to Do Addition](https://mlsanity.com/paper/2502.00873/) (2025)

## Built on by (most cited first, 30 of 31)

- [Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds](https://mlsanity.com/paper/2606.25234/) (2026)
- [Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability](https://mlsanity.com/paper/2606.06333/) (2026)
- [Closure-Validated Circuit Discovery in Attention Heads: Co-activation Proposes, Ablation Disposes](https://mlsanity.com/paper/2606.09607/) (2026)
- [Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal](https://mlsanity.com/paper/2606.12360/) (2026)
- [Bilinear autoencoders find interpretable manifolds](https://mlsanity.com/paper/2605.08891/) (2026)
- [A Geometric View for Understanding Concept Learning and Neuron Interpretation in Sparse Autoencoders](https://mlsanity.com/paper/2606.07007/) (2026)
- [Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders](https://mlsanity.com/paper/2606.12138/) (2026)
- [DiMaS: Distribution Matching for Steering Vision-Language-Action Models](https://mlsanity.com/paper/2607.14280/) (2026)
- [Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects](https://mlsanity.com/paper/2607.24645/) (2026)
- [The Emergent Symbolic Structure of Artificial Neural Networks](https://mlsanity.com/paper/2608.29530/) (2026)
- [Capsule Lens: Locating and Tracking Concept Geometry in Model Representations](https://mlsanity.com/paper/2609.05575/) (2026)
- [How Far Does a Shared Linear Map Go? Probing Feature-Space Manipulability for Image Editing](https://mlsanity.com/paper/2605.11203/) (2026)
- [Relational Rank Geometry in Transformers: Detecting and Steering Hidden-State Relation Frames](https://mlsanity.com/paper/2605.29634/) (2026)
- [The Importance of Phase in Neural Representations: An Internal Oppenheim-Lim Test of Image Classifiers](https://mlsanity.com/paper/2606.17037/) (2026)
- [Subliminal Clocks: Latent Time Modelling in Diffusion Language Models](https://mlsanity.com/paper/2607.01774/) (2026)
- [Laguerre Geometry for Interpreting Large Language Models](https://mlsanity.com/paper/2607.10578/) (2026)
- [Explaining Image Similarity with Automatically Extracted Concept Activation Vectors](https://mlsanity.com/paper/2607.28386/) (2026)
- [A Deeper Analysis of Block-Sparse Featurizers](https://mlsanity.com/paper/2608.27515/) (2026)
- [Efficient Auto-Interpretability of AI Models in Biology](https://mlsanity.com/paper/2608.27754/) (2026)
- [MAxBench: A Multinomial Concept Recovery Benchmark](https://mlsanity.com/paper/2609.13072/) (2026)
- [Xeno-Interpretability: Investigating the Alien Minds of LLMs](https://mlsanity.com/paper/2609.20408/) (2026)
- [Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoders](https://mlsanity.com/paper/2609.31056/) (2026)
- [Gradients for Interventions and Activations for Detection: Targeted Feature Learning in Language Models](https://mlsanity.com/paper/2609.32355/) (2026)
- [Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?](https://mlsanity.com/paper/2609.35020/) (2026)
- [When Models Don't Manipulate Manifolds: The Geometry of a Comparison Task](https://mlsanity.com/paper/2609.37680/) (2026)
- [Temporally-Resolved Token Attribution Reveals the Generation Dynamics of Diffusion Language Models](https://mlsanity.com/paper/2610.01177/) (2026)
- [The Independence Prior of SAEs Fragments Visual Concepts](https://mlsanity.com/paper/2610.04112/) (2026)
- [A Testable Theory of Atomic Features](https://mlsanity.com/paper/2610.05794/) (2026)
- [IntactWorld: Joint World Modeling with Intact Features](https://mlsanity.com/paper/2610.11174/) (2026)
- [RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing](https://mlsanity.com/paper/2610.11775/) (2026)

## Abstract (arXiv)

Sparse autoencoders (SAEs) are widely used to extract interpretable features from neural network representations, often under the implicit assumption that concepts correspond to independent linear directions. However, a growing body of evidence suggests that many concepts are instead organized along low-dimensional manifolds encoding continuous geometric relationships. This raises three basic questions: what does it mean for an SAE to capture a manifold, when do existing SAE architectures do so, and how? We develop a theoretical framework that answers these questions and show that SAEs can capture manifolds in two fundamentally different ways: globally, by allocating a compact group of atoms whose linear span contains the entire manifold, or locally, by distributing it across features that each selectively tile a restricted region of the underlying geometry. Empirically, we find that SAEs suboptimally recover continuous structures, mixing the global subspace and local tiling solutions in a fragmented regime we call dilution. This explains why manifold structure is rarely visible at the level of individual concepts and motivates post-hoc unsupervised discovery methods that search for coherent groups of atoms rather than isolated directions. More broadly, our results suggest that future representation learning methods should treat geometric objects, not just individual directions, as the basic units of interpretability.