mlsanityData to 7 Oct 2026

Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2

arXiv 2408.05147 ↗First posted 9 Aug 2024cs.LG cs.AI cs.CLcited by 381

Not on the map. Why?

Cited by 26 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Sparse autoencoders (SAEs) are an unsupervised method for learning a sparse decomposition of a neural network's latent representations into seemingly interpretable features. Despite recent excitement about their potential, research applications outside of industry are limited by the high cost of training a comprehensive suite of SAEs. In this work, we introduce Gemma Scope, an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2 2B and 9B and select layers of Gemma 2 27B base models. We primarily train SAEs on the Gemma 2 pre-trained models, but additionally release SAEs trained on instruction-tuned Gemma 2 9B for comparison. We evaluate the quality of each SAE on standard metrics and release these results. We hope that by releasing these SAE weights, we can help make more ambitious safety and interpretability research easier for the community. Weights and a tutorial can be found at https://huggingface.co/google/gemma-scope and an interactive demo can be found at https://www.neuronpedia.org/gemma-scope

Key facts. Cited by 381 papers in the map (data through 7 Oct 2026). Builds on 37 of its 94 references that are in the map. First posted on arXiv 9 Aug 2024.

Builds on

37 of 94 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  8. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  9. 2019Root Mean Square Layer Normalizationcited by 1,425
  10. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  12. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
25 more
  1. 2022In-context Learning and Induction Headscited by 930
  2. 2022Toy Models of Superpositioncited by 928
  3. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  5. 2024Scaling and evaluating sparse autoencoderscited by 530
  6. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  7. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  8. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  9. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  10. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  11. 2023In-Context Learning Creates Task Vectorscited by 302
  12. 2023Function Vectors in Large Language Modelscited by 292
  13. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  14. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  15. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 186
  16. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  17. 2024Universal Neurons in GPT2 Language Modelscited by 100
  18. 2023A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysiscited by 88
  19. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 83
  20. 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 64
  21. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  22. 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Studycited by 54
  23. 2024Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Modelscited by 51
  24. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
  25. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 35; 2025: 164; 2026: 182.

381 papers.

Sort:

By area:

  1. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  2. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  3. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 136
  4. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 133
  5. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  6. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
  7. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  8. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  9. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
  10. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 86
  11. 2025Persona Features Control Emergent Misalignmentcited by 76
  12. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
38 more
  1. 2024Sparse autoencoders reveal selective remapping of visual concepts during adaptationcited by 66
  2. 2024Applying sparse autoencoders to unlearn knowledge in language modelscited by 58
  3. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
  4. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
  5. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  6. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 49
  7. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 45
  8. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 43
  9. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
  10. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  11. 2024Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Smallcited by 34
  12. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 34
  13. 2025LLM Pretraining with Continuous Conceptscited by 33
  14. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additionscited by 32
  15. 2024One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Modelscited by 30
  16. 2024Towards Unifying Interpretability and Control: Evaluation via Interventioncited by 28
  17. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 28
  18. 2025Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Propertiescited by 28
  19. 2025Training Language Models to Explain Their Own Computationscited by 27
  20. 2025Towards Understanding Distilled Reasoning Models: A Representational Approachcited by 26
  21. 2025Understanding Refusal in Language Models with Sparse Autoencoderscited by 26
  22. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 25
  23. 2025Route Sparse Autoencoder to Interpret Large Language Modelscited by 25
  24. 2025How Visual Representations Map to Language Feature Space in Multimodal LLMscited by 24
  25. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  26. 2024Sparsing Law: Towards Large Language Models with Greater Activation Sparsitycited by 22
  27. 2025Mixture of Experts Made Intrinsically Interpretablecited by 22
  28. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  29. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
  30. 2024Mechanistic Permutability: Match Features Across Layerscited by 21
  31. 2025Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoderscited by 21
  32. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 21
  33. 2025HyperSteer: Activation Steering at Scale with Hypernetworkscited by 20
  34. 2024Can sparse autoencoders be used to decompose and interpret steering vectors?cited by 19
  35. 2025Sparse Autoencoder Features for Classifications and Transferabilitycited by 19
  36. 2025Towards eliciting latent knowledge from LLMs with mechanistic interpretabilitycited by 19
  37. 2024Residual Stream Analysis with Multi-Layer SAEscited by 18
  38. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18

50 most cited of 381 All 381 (JSON)