mlsanityData to 7 Oct 2026

Scaling and evaluating sparse autoencoders

arXiv 2406.04093 ↗First posted 6 Jun 2024cs.LG cs.AIcited by 530

Not on the map. Why?

Cited by 34 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Sparse autoencoders provide a promising unsupervised approach for extracting interpretable features from a language model by reconstructing activations from a sparse bottleneck layer. Since language models learn many concepts, autoencoders need to be very large to recover all relevant features. However, studying the properties of autoencoder scaling is difficult due to the need to balance reconstruction and sparsity objectives and the presence of dead latents. We propose using k-sparse autoencoders [Makhzani and Frey, 2013] to directly control sparsity, simplifying tuning and improving the reconstruction-sparsity frontier. Additionally, we find modifications that result in few dead latents, even at the largest scales we tried. Using these techniques, we find clean scaling laws with respect to autoencoder size and sparsity. We also introduce several new metrics for evaluating feature quality based on the recovery of hypothesized features, the explainability of activation patterns, and the sparsity of downstream effects. These metrics all generally improve with autoencoder size. To demonstrate the scalability of our approach, we train a 16 million latent autoencoder on GPT-4 activations for 40 billion tokens. We release training code and autoencoders for open-source models, as well as a visualizer.

Key facts. Cited by 530 papers in the map (data through 7 Oct 2026). Builds on 35 of its 69 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

35 of 69 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  10. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
23 more
  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  2. 2022Toy Models of Superpositioncited by 928
  3. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  4. 2020Aligning AI With Shared Human Valuescited by 706
  5. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  6. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  7. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  8. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  9. 2024Massive Activations in Large Language Modelscited by 277
  10. 2022Unified Scaling Laws for Routed Language Modelscited by 249
  11. 2013k-Sparse Autoencoderscited by 209
  12. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  13. 2019"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understandingcited by 121
  14. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 108
  15. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 106
  16. 2021An Interpretability Illusion for BERTcited by 94
  17. 2019QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questionscited by 89
  18. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 83
  19. 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 64
  20. 2014Sparse Modeling for Image and Vision Processingcited by 48
  21. 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 26
  22. 2014Zero-bias autoencoders and the benefits of co-adapting featurescited by 15
  23. 2019JumpReLU: A Retrofit Defense Strategy for Adversarial Attackscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 55; 2025: 210; 2026: 265.

530 papers.

Sort:

By area:

  1. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  2. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  3. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  4. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  5. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 136
  6. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 133
  7. 2024BatchTopK Sparse Autoencoderscited by 126
  8. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  9. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
  10. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  11. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  12. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
38 more
  1. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  2. 2025Persona Features Control Emergent Misalignmentcited by 76
  3. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  4. 2025Sparse Autoencoders Learn Monosemantic Features in Vision-Language Modelscited by 70
  5. 2025Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Modelscited by 67
  6. 2025Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignmentcited by 62
  7. 2024Applying sparse autoencoders to unlearn knowledge in language modelscited by 58
  8. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
  9. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 56
  10. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
  11. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  12. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
  13. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  14. 2024Scaling Laws for Predicting Downstream Performance in LLMscited by 48
  15. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  16. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
  17. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 43
  18. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
  19. 2025How Do Large Language Monkeys Get Their Power (Laws)?cited by 40
  20. 2025Weight-sparse transformers have interpretable circuitscited by 40
  21. 2024Efficient Dictionary Learning with Switch Sparse Autoencoderscited by 38
  22. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  23. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?cited by 35
  24. 2024Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robustcited by 33
  25. 2025LLM Pretraining with Continuous Conceptscited by 33
  26. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  27. 2025Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuningcited by 31
  28. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  29. 2024One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Modelscited by 30
  30. 2024textitRevelio: Interpreting and leveraging semantic information in diffusion modelscited by 29
  31. 2024Out-of-distribution generalization via composition: a lens through induction heads in Transformerscited by 28
  32. 2024Bilinear MLPs enable weight-based mechanistic interpretabilitycited by 28
  33. 2024Towards Unifying Interpretability and Control: Evaluation via Interventioncited by 28
  34. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  35. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 25
  36. 2024Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoderscited by 25
  37. 2025Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representationcited by 25
  38. 2025Route Sparse Autoencoder to Interpret Large Language Modelscited by 25

50 most cited of 530 All 530 (JSON)