mlsanityData to 7 Oct 2026

A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models

arXiv 2503.05613 ↗First posted 7 Mar 2025cs.LG cs.AI cs.CLcited by 73

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have transformed natural language processing, yet their internal mechanisms remain largely opaque. Recently, mechanistic interpretability has attracted significant attention from the research community as a means to understand the inner workings of LLMs. Among various mechanistic interpretability approaches, Sparse Autoencoders (SAEs) have emerged as a promising method due to their ability to disentangle the complex, superimposed features within LLMs into more interpretable components. This paper presents a comprehensive survey of SAEs for interpreting and understanding the internal workings of LLMs. Our major contributions include: (1) exploring the technical framework of SAEs, covering basic architecture, design improvements, and effective training strategies; (2) examining different approaches to explaining SAE features, categorized into input-based and output-based explanation methods; (3) discussing evaluation methods for assessing SAE performance, covering both structural and functional metrics; and (4) investigating real-world applications of SAEs in understanding and manipulating LLM behaviors.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 49 of its 85 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

49 of 85 references.

References · 49

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  8. 2020GLU Variants Improve Transformercited by 1,724
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  10. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  11. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  12. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
37 more
  1. 2016Language Modeling with Gated Convolutional Networkscited by 688
  2. 2018Disentangling by Factorisingcited by 614
  3. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  4. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  5. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  6. 2018Towards a Definition of Disentangled Representationscited by 294
  7. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  8. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 250
  9. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  10. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  11. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  12. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  13. 2017Learning to Generate Reviews and Discovering Sentimentcited by 147
  14. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 133
  15. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  16. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  17. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  18. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 86
  19. 2019Understanding Neural Networks via Feature Visualization: A surveycited by 65
  20. 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 64
  21. 2024Applying sparse autoencoders to unlearn knowledge in language modelscited by 58
  22. 2017SPINE: SParse Interpretable Neural Embeddingscited by 55
  23. 2015Sparse Overcomplete Word Vector Representationscited by 43
  24. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  25. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 32
  26. 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 26
  27. 2024Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoderscited by 25
  28. 2024Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distributioncited by 24
  29. 2023A Geometric Notion of Causal Probingcited by 23
  30. 2024Towards Uncovering How Large Language Model Works: An Explainability Perspectivecited by 22
  31. 2025Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoderscited by 21
  32. 2025Sparse Autoencoder Features for Classifications and Transferabilitycited by 19
  33. 2024Analyzing (In)Abilities of SAEs via Formal Languagescited by 16
  34. 2024Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Modelscited by 15
  35. 2023The Importance of Prompt Tuning for Automated Neuron Explanationscited by 12
  36. 2024Group-SAE: Efficient Training of Sparse Autoencoders for Large Language Models via Layer Groupscited by 10
  37. 2024Adaptive Sparse Allocation with Mutual Choice & Feature Choice Sparse Autoencoderscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 33; 2026: 40.

73 papers.

Sort:

By area:

  1. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 43
  2. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  3. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 15
  4. 2025Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Modelscited by 12
  5. 2025Measuring and Guiding Monosemanticitycited by 10
  6. 2025SoK: Machine Unlearning for Large Language Modelscited by 9
  7. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinementcited by 9
  8. 2025OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Featurescited by 7
  9. 2025Rep2Text: Decoding Full Text from a Single LLM Token Representationcited by 7
  10. 2025When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retractioncited by 6
  11. 2025Rethinking Explainability in the Era of Multimodal AIcited by 6
  12. 2025Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoderscited by 6
38 more
  1. 2026Position: Anthropomorphic Misalignment Research Needs Stronger Evidencecited by 6
  2. 2025AlignSAE: Concept-Aligned Sparse Autoencoderscited by 4
  3. 2026TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steeringcited by 4
  4. 2026Monitoring the Internal Monologue: Probe Trajectories Reveal Reasoning Dynamicscited by 4
  5. 2026SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentationcited by 4
  6. 2025Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influencecited by 3
  7. 2025Mechanistic Interpretability with SAEs: Probing Religion, Violence, and Geography in Large Language Modelscited by 3
  8. 2025Toward Preference-aligned Large Language Models via Residual-based Model Steeringcited by 3
  9. 2026Interpreting and Steering State-Space Models via Activation Subspace Bottleneckscited by 3
  10. 2026How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularitiescited by 3
  11. 2025Evaluating Steering Techniques using Human Similarity Judgmentscited by 2
  12. 2025A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minimacited by 2
  13. 2026NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Modelscited by 2
  14. 2026Towards Interpretable Hallucination Analysis and Mitigation in LVLMs via Contrastive Neuron Steeringcited by 2
  15. 2026Mechanistic Interpretability of Cognitive Complexity in LLMs via Linear Probing using Bloom's Taxonomycited by 2
  16. 2026Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modelingcited by 2
  17. 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Modelscited by 2
  18. 2026ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotationcited by 2
  19. 2026Evaluating the Interpretability of Sparse Autoencoders with Concept Annotationscited by 2
  20. 2025SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selectioncited by 1
  21. 2025Even Heads Fix Odd Errors: Mechanistic Discovery and Surgical Repair in Transformer Attentioncited by 1
  22. 2025ConceptViz: A Visual Analytics Approach for Exploring Concepts in Large Language Modelscited by 1
  23. 2025Binary Autoencoder for Mechanistic Interpretability of Large Language Modelscited by 1
  24. 2025AI Alignment Strategies from a Risk Perspective: Independent Safety Mechanisms or Shared Failures?cited by 1
  25. 2025Feature-Guided SAE Steering for Refusal-Rate Control using Contrasting Promptscited by 1
  26. 2025Beyond Redundancy: Diverse and Specialized Multi-Expert Sparse Autoencodercited by 1
  27. 2026Event-Grounded Sparse Autoencoders for Vision-Language-Action Policiescited by 1
  28. 2026Bug or Feature2: Weight Drift, Activation Sparsity and Spikescited by 1
  29. 2026Do Sparse Autoencoders Learn Meaningful Concept Hierarchies?cited by 1
  30. 2026What's in an Earth Embedding? An Explainability Analysis of Location Encoderscited by 1
  31. 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoningcited by 1
  32. 2025Concept-Guided Interpretability via Neural Chunkingcited by 0
  33. 2025Sparsification and Reconstruction from the Perspective of Representation Geometrycited by 0
  34. 2025TopK Language Modelscited by 0
  35. 2025Disentangling concept semantics via multilingual averaging in Sparse Autoencoderscited by 0
  36. 2025VOX-KRIKRI: Unifying Speech and Language through Continuous Fusioncited by 0
  37. 2025Concept-SAE: A Controllable and Invertible Concept Interface for Sparse Autoencoderscited by 0
  38. 2025Language over Content: Tracing Cultural Understanding in Multilingual Large Language Modelscited by 0

50 most cited of 73 All 73 (JSON)