# Controllable LLM Reasoning via Sparse Autoencoder-Based Steering

arXiv: https://arxiv.org/abs/2601.03595 · HTML: https://mlsanity.com/paper/2601.03595/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 16 of 16)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs](https://mlsanity.com/paper/2501.18585/) (2025)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [DeepSeek-R1 Thoughtology: Let's think about LLM Reasoning](https://mlsanity.com/paper/2504.07128/) (2025)
- [Effectively Controlling Reasoning Models through Thinking Intervention](https://mlsanity.com/paper/2503.24370/) (2025)
- [Trading Inference-Time Compute for Adversarial Robustness](https://mlsanity.com/paper/2501.18841/) (2025)
- [I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2503.18878/) (2025)
- [A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Law](https://mlsanity.com/paper/2505.02665/) (2025)
- [LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Models](https://mlsanity.com/paper/2501.11036/) (2025)

## Built on by (most cited first, 15 of 15)

- [Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Models](https://mlsanity.com/paper/2605.11887/) (2026)
- [Do Sparse Autoencoders Identify Reasoning Features in Language Models?](https://mlsanity.com/paper/2601.05679/) (2026)
- [Interpreting and Controlling LLM Reasoning through Integrated Policy Gradient](https://mlsanity.com/paper/2602.02313/) (2026)
- [The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models](https://mlsanity.com/paper/2606.06188/) (2026)
- [Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models](https://mlsanity.com/paper/2605.22902/) (2026)
- [Steered Generation via Gradient-Based Optimization on Sparse Query Features](https://mlsanity.com/paper/2605.23040/) (2026)
- [Discovering Millions of Interpretable Features with Sparse Autoencoders](https://mlsanity.com/paper/2606.26620/) (2026)
- [SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models](https://mlsanity.com/paper/2607.19364/) (2026)
- [Scaling Interpretable Transformers with Parity Bottleneck Layers](https://mlsanity.com/paper/2607.20652/) (2026)
- [Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering](https://mlsanity.com/paper/2607.26411/) (2026)
- [Measuring Semantic Abstractness of SAE Features via Nonlocality](https://mlsanity.com/paper/2608.10537/) (2026)
- [Answer Probing-Guided Search for Diverse Solution Exploration of LLMs](https://mlsanity.com/paper/2608.30345/) (2026)
- [First-Order Steering: Translating Weight Adaptation into Activation Steering](https://mlsanity.com/paper/2610.04283/) (2026)
- [SelectTTRL: Representation-Guided Data Selection for Efficient Test-Time Reinforcement Learning](https://mlsanity.com/paper/2610.16104/) (2026)
- [Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility](https://mlsanity.com/paper/2610.20139/) (2026)

## Abstract (arXiv)

Large Reasoning Models (LRMs) exhibit human-like cognitive reasoning strategies (eg backtracking, cross-verification) during the reasoning process, which improves their performance on complex tasks. Currently, reasoning strategies are autonomously selected by LRMs themselves. However, such autonomous selection often produces inefficient or even erroneous reasoning paths. To make reasoning more reliable and flexible, it is important to develop methods for controlling reasoning strategies. Existing methods struggle to control fine-grained reasoning strategies due to conceptual entanglement in LRMs' hidden states. To address this, we leverage Sparse Autoencoders (SAEs) to decompose strategy-entangled hidden states into a disentangled feature space. To identify the few strategy-specific features from the vast pool of SAE features, we propose SAE-Steering, an efficient two-stage feature identification pipeline. SAE-Steering first recalls features that amplify the logits of strategy-specific keywords, filtering out over 99% of features, and then ranks the remaining features by their control effectiveness. Using the identified strategy-specific features as control vectors, SAE-Steering outperforms existing methods by over 15% in control effectiveness. Furthermore, controlling reasoning strategies can redirect LRMs from erroneous paths to correct ones, achieving a 7% absolute accuracy improvement. Our code and data are available at https://github.com/Peter-Fy/SAE-Steering.