# Sparse autoencoder training and evaluation

Area: [Mechanistic interpretability](https://mlsanity.com/mechinterp/index.md). HTML: https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/

## Key facts

- Sparse autoencoder training and evaluation: 175 papers on arXiv since 2021 (data through 14 Oct 2026).
- 84 of its 175 mapped papers (48%) cite “Toy Models of Superposition” (2022, arXiv 2209.10652).
- 42 of its 175 mapped papers (24%) cite “Learning Multi-Level Features with Matryoshka Sparse Autoencoders” (2025, arXiv 2503.17547).
- 15 new papers 24 Sep–14 Oct.

## Most cited here

- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022), cited by 84 papers here
- [Learning Multi-Level Features with Matryoshka Sparse Autoencoders](https://mlsanity.com/paper/2503.17547/) (2025), cited by 42 papers here
- [Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2](https://mlsanity.com/paper/2408.05147/) (2024), cited by 40 papers here
- [Sparse Autoencoders Do Not Find Canonical Units of Analysis](https://mlsanity.com/paper/2502.04878/) (2025), cited by 25 papers here
- [Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models](https://mlsanity.com/paper/2502.12892/) (2025), cited by 21 papers here
- [Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry](https://mlsanity.com/paper/2503.01822/) (2025), cited by 19 papers here
- [Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning](https://mlsanity.com/paper/2405.12241/) (2024), cited by 17 papers here
- [From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit](https://mlsanity.com/paper/2506.03093/) (2025), cited by 16 papers here
- [Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control](https://mlsanity.com/paper/2405.08366/) (2024), cited by 15 papers here
- [The Geometry of Concepts: Sparse Autoencoder Feature Structure](https://mlsanity.com/paper/2410.19750/) (2024), cited by 11 papers here

## New (24 Sep–14 Oct)

- [Complete or Sparse: A Tale of Two Identifiabilities](https://mlsanity.com/paper/2610.20851/) (2026-10-14)
- [Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability](https://mlsanity.com/paper/2610.19907/) (2026-10-14)
- [Readout Scaling Shapes Sparse Feature Recovery](https://mlsanity.com/paper/2610.18462/) (2026-10-13)
- [Rescuing Points Trapped Inside Incorrect Decision Regions: A target for Evaluating and Improving Forward-Forward Learning](https://mlsanity.com/paper/2610.16347/) (2026-10-12)
- [Small area learning](https://mlsanity.com/paper/2610.15953/) (2026-10-11)
- [What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions](https://mlsanity.com/paper/2610.15850/) (2026-10-11)
- [Inference and learning in sparse autoencoders as natural gradient flow](https://mlsanity.com/paper/2610.07389/) (2026-10-05)
- [ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection](https://mlsanity.com/paper/2610.04905/) (2026-10-04)
- [The Independence Prior of SAEs Fragments Visual Concepts](https://mlsanity.com/paper/2610.04112/) (2026-10-02)
- [ClasSAE: Class-Aligned Sparse Autoencoders via Differentiable Feature-Class Affinity](https://mlsanity.com/paper/2610.04020/) (2026-10-02)
- [Whitening Improves Robustness to Spurious Correlations in Linear Probes](https://mlsanity.com/paper/2609.39177/) (2026-09-30)
- [Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?](https://mlsanity.com/paper/2609.35020/) (2026-09-28)
- [When Is an SAE Feature Interpretable? A Validation Ladder for EEG Foundation Models](https://mlsanity.com/paper/2609.34091/) (2026-09-28)
- [PULSE: Identifying Demonstration-Utility Features with Sparse Autoencoders](https://mlsanity.com/paper/2609.32469/) (2026-09-26)
- [Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoders](https://mlsanity.com/paper/2609.31056/) (2026-09-25)