mlsanityData to 14 Oct 2026

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders

arXiv 2410.20526 ↗First posted 27 Oct 2024cs.LG cs.CLcited by 143

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

Sparse Autoencoders (SAEs) have emerged as a powerful unsupervised method for extracting sparse representations from language models, yet scalable training remains a significant challenge. We introduce a suite of 256 SAEs, trained on each layer and sublayer of the Llama-3.1-8B-Base model, with 32K and 128K features. Modifications to a state-of-the-art SAE variant, Top-K SAEs, are evaluated across multiple dimensions. In particular, we assess the generalizability of SAEs trained on base models to longer contexts and fine-tuned models. Additionally, we analyze the geometry of learned SAE latents, confirming that feature splitting enables the discovery of new features. The Llama Scope SAE checkpoints are publicly available at~urlhttps://huggingface.co/fnlp/Llama-Scope, alongside our scalable training, interpretation, and visualization tools at urlhttps://github.com/OpenMOSS/Language-Model-SAEs. These contributions aim to advance the open-source Sparse Autoencoder ecosystem and support mechanistic interpretability research by reducing the need for redundant SAE training.

Key facts. Cited by 143 papers in the map (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 27 of its 42 references that are in the map. First posted on arXiv 27 Oct 2024.

Builds on

27 of 42 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoders9 Oct
  2. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders19 Jul
  3. 2024Scaling and evaluating sparse autoencoders6 Jun
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models15 Sep

Other references · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  6. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  7. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  8. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  9. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  10. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  11. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 233
  12. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
11 more
  1. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
  2. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  3. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  4. 2024Universal Neurons in GPT2 Language Modelscited by 102
  5. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  6. 2024Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representationscited by 47
  7. 2024Efficient Dictionary Learning with Switch Sparse Autoencoderscited by 39
  8. 2024Towards Universality: Studying Mechanistic Similarity Across Language Model Architecturescited by 31
  9. 2024Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPTcited by 29
  10. 2024Automatically Identifying Local and Global Circuits with Linear Computation Graphscited by 24
  11. 2019JumpReLU: A Retrofit Defense Strategy for Adversarial Attackscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 66; 2026: 76.

143 papers.

Sort:

By area:

  1. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
  2. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 78
  3. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  4. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 56
  5. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 41
  6. 2025Understanding the Repeat Curse in Large Language Models from a Feature Perspectivecited by 37
  7. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 34
  8. 2025Training Language Models to Explain Their Own Computationscited by 30
  9. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  10. 2025Route Sparse Autoencoder to Interpret Large Language Modelscited by 26
  11. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
  12. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24
38 more
  1. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
  2. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 18
  3. 2025Interpretable Reward Model via Sparse Autoencodercited by 18
  4. 2025Feature-Level Insights into Artificial Text Detection with Sparse Autoencoderscited by 15
  5. 2025Visual Representations inside the Language Modelcited by 15
  6. 2025Evaluation Faking: Unveiling Observer Effects in Safety Evaluation of Frontier AI Systemscited by 14
  7. 2025Analyze Feature Flow to Enhance Interpretation and Steering in Language Modelscited by 13
  8. 2025Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoderscited by 13
  9. 2025Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Modelscited by 13
  10. 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decodingcited by 13
  11. 2025Improving LLM Reasoning through Interpretable Role-Playing Steeringcited by 12
  12. 2025CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Featurescited by 12
  13. 2025Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistantscited by 12
  14. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 12
  15. 2026From Directions to Regions: Decomposing Activations in Language Models via Local Geometrycited by 12
  16. 2025Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metriccited by 11
  17. 2025Precise In-Parameter Concept Erasure in Large Language Modelscited by 11
  18. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  19. 2025Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Frameworkcited by 10
  20. 2025Do Code LLMs Understand Design Patterns?cited by 9
  21. 2025Towards Understanding the Nature of Attention with Low-Rank Sparse Decompositioncited by 9
  22. 2025Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Modelscited by 9
  23. 2025Evaluating and Designing Sparse Autoencoders by Approximating Quasi-Orthogonalitycited by 8
  24. 2025HuggingGraph: Understanding the Supply Chain of LLM Ecosystemcited by 8
  25. 2025Evolution of Concepts in Language Model Pre-Trainingcited by 8
  26. 2025Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Trainingcited by 8
  27. 2026AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoderscited by 8
  28. 2026Learning a Generative Meta-Model of LLM Activationscited by 8
  29. 2025Low-Rank Adapting Models for Sparse Autoencoderscited by 7
  30. 2025SAEMark: Steering Personalized Multilingual LLM Watermarks with Sparse Autoencoderscited by 7
  31. 2025SparseRM: A Lightweight Preference Modeling with Sparse Autoencodercited by 7
  32. 2026Fine-Grained Activation Steering: Steering Less, Achieving Morecited by 7
  33. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 7
  34. 2026Understanding Emergent Misalignment via Feature Superposition Geometrycited by 7
  35. 2025AI shares emotion with humans across languages and culturescited by 6
  36. 2025Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learningcited by 6
  37. 2025Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Frameworkcited by 6
  38. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6

50 most cited of 143 All 143 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse Autoencoders Trained on the Same Data Learn Different Features
  2. 2024Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment
  3. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
  4. 2024Improving Dictionary Learning with Gated Sparse Autoencoders
  5. 2025Sparse Autoencoder Features for Classifications and Transferability
  6. 2026Steering Vision-Language Models with Joint Sparse Autoencoders
  7. 2026When Can We Trust the Sparse Lens? A Certification Framework for SAE Faithfulness
  8. 2025FaithfulSAE: Towards Capturing Faithful Features with Sparse Autoencoders without External Dataset Dependencies