mlsanityData to 7 Oct 2026

I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders

arXiv 2503.18878 ↗First posted 24 Mar 2025cs.CLcited by 43

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recent LLMs like DeepSeek-R1 have demonstrated state-of-the-art performance by integrating deep thinking and complex reasoning during generation. However, the internal mechanisms behind these reasoning processes remain unexplored. We observe reasoning LLMs consistently use vocabulary associated with human reasoning processes. We hypothesize these words correspond to specific reasoning moments within the models' internal mechanisms. To test this hypothesis, we employ Sparse Autoencoders (SAEs), a technique for sparse decomposition of neural network activations into human-interpretable features. We introduce ReasonScore, an automatic metric to identify active SAE features during these reasoning moments. We perform manual and automatic interpretation of the features detected by our metric, and find those with activation patterns matching uncertainty, exploratory thinking, and reflection. Through steering experiments, we demonstrate that amplifying these features increases performance on reasoning-intensive benchmarks (+2.2%) while producing longer reasoning traces (+20.5%). Using the model diffing technique, we provide evidence that these features are present only in models with reasoning capabilities. Our work provides the first step towards a mechanistic understanding of reasoning in LLMs. Code available at https://github.com/AIRI-Institute/SAE-Reasoning

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 32 of its 64 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

32 of 64 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  11. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  12. 2025s1: Simple test-time scalingcited by 1,311
20 more
  1. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  2. 2022Toy Models of Superpositioncited by 928
  3. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  4. 2024Scaling and evaluating sparse autoencoderscited by 530
  5. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  6. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  7. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  8. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  9. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  10. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
  11. 2024On the Origins of Linear Representations in Large Language Modelscited by 76
  12. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 75
  13. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 73
  14. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additionscited by 32
  15. 2025Route Sparse Autoencoder to Interpret Large Language Modelscited by 25
  16. 2024Mechanistic Permutability: Match Features Across Layerscited by 21
  17. 2023Is Bigger and Deeper Always Better? Probing LLaMA Across Scales and Layerscited by 20
  18. 2025Feature-Level Insights into Artificial Text Detection with Sparse Autoencoderscited by 14
  19. 2024Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspectivecited by 10
  20. 2024Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers?cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 28.

43 papers.

Sort:

By area:

  1. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 25
  2. 2025Reasoning-Finetuning Repurposes Latent Representations in Base Modelscited by 23
  3. 2025Get Experience from Practice: LLM Agents with Record & Replaycited by 22
  4. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  5. 2025On Predictability of Reinforcement Learning Dynamics for Large Language Modelscited by 20
  6. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safetycited by 18
  7. 2026NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neuronscited by 13
  8. 2025Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Modelscited by 12
  9. 2026Controllable LLM Reasoning via Sparse Autoencoder-Based Steeringcited by 12
  10. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 11
  11. 2026Reasoning Models Generate Societies of Thoughtcited by 11
  12. 2026Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?cited by 10
31 more
  1. 2025KV Cache Steering for Controlling Frozen LLMscited by 8
  2. 2026Good Arguments Against the People Pleasers: How Reasoning Mitigates (Yet Masks) LLM Sycophancycited by 7
  3. 2025Meta-R1: Empowering Large Reasoning Models with Metacognitioncited by 6
  4. 2025From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Modelscited by 6
  5. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 5
  6. 2026Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steeringcited by 4
  7. 2026What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Featurescited by 4
  8. 2026SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentationcited by 4
  9. 2025Beyond Input Activations: Identifying Influential Latents by Gradient Sparse Autoencoderscited by 3
  10. 2025Mechanistic Interpretability with SAEs: Probing Religion, Violence, and Geography in Large Language Modelscited by 3
  11. 2025A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steeringcited by 3
  12. 2026Interpreting and Controlling LLM Reasoning through Integrated Policy Gradientcited by 3
  13. 2025SASFT: Sparse Autoencoder-guided Supervised Finetuning to Mitigate Unexpected Code-Switching in LLMscited by 2
  14. 2026Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanismscited by 2
  15. 2026Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Modelscited by 2
  16. 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Modelscited by 2
  17. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  18. 2026Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarkscited by 1
  19. 2026Mechanistically Interpreting the Role of Sample Difficulty in RLVR for LLMscited by 1
  20. 2025Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoringcited by 0
  21. 2026Beyond Activation Patterns: A Weight-Based Out-of-Context Explanation of Sparse Autoencoder Featurescited by 0
  22. 2026TRACES: Tagging Reasoning Steps for Adaptive Cost-Efficient Early-Stoppingcited by 0
  23. 2026Steered Generation via Gradient-Based Optimization on Sparse Query Featurescited by 0
  24. 2026PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoningcited by 0
  25. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  26. 2026Cross-seed explainability using Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoderscited by 0
  27. 2026Uncovering Latent Reasoning Strategies in Language Modelscited by 0
  28. 2026Measuring Semantic Abstractness of SAE Features via Nonlocalitycited by 0
  29. 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMscited by 0
  30. 2026Same Values, Different Languages? From Multilingual Probing to Steering LLMs Toward Chinese Social Valuescited by 0
  31. 2026Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Step-Level Sparse Autoencoder for Reasoning Process Interpretation
  2. 2026Enhancing Low-Resource Language Reasoning via High-Resource Language Feature Transfer
  3. 2026Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders
  4. 2023Large Language Models Are Not Strong Abstract Reasoners
  5. 2025Exploring Task Performance with Interpretable Models via Sparse Auto-Encoders
  6. 2025ActivationReasoning: Logical Reasoning in Latent Activation Spaces
  7. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
  8. 2025Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process