mlsanityData to 14 Oct 2026

Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models

arXiv 2505.15634 ↗First posted 21 May 2025cs.CL cs.LGcited by 13

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) demonstrate the ability to solve reasoning and mathematical problems using the Chain-of-Thought (CoT) technique. Expanding CoT length, as seen in models such as DeepSeek-R1, significantly enhances this reasoning for complex problems, but requires costly and high-quality long CoT data and fine-tuning. This work, inspired by the deep thinking paradigm of DeepSeek-R1, utilizes a steering technique to enhance the reasoning ability of an LLM without external datasets. Our method first employs Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT. These features are then used to steer the LLM's internal states during generation. Recognizing that many LLMs do not have corresponding pre-trained SAEs, we further introduce a novel SAE-free steering algorithm, which directly computes steering directions from the residual activations of an LLM, obviating the need for an explicit SAE. Experimental results demonstrate that both our SAE-based and subsequent SAE-free steering algorithms significantly enhance the reasoning capabilities of LLMs.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 52 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

38 of 52 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering14 Mar
  2. 2025Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models14 Mar
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 35

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  9. 2024OpenAI o1 System Cardcited by 1,244
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  11. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  12. 2024Scaling and evaluating sparse autoencoderscited by 553
23 more
  1. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  2. 2025Humanity's Last Examcited by 491
  3. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  4. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  5. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  6. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  7. 2023Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learningcited by 236
  8. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 179
  9. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
  10. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  11. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 63
  12. 2024Applying sparse autoencoders to unlearn knowledge in language modelscited by 59
  13. 2025I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoderscited by 45
  14. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 34
  15. 2025FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Modelscited by 27
  16. 2025Improving Reasoning Performance in Large Language Models via Representation Engineeringcited by 23
  17. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 22
  18. 2025BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoningcited by 21
  19. 2025Interpreting and Steering LLMs with Mutual Information-based Explanations on Sparse Autoencoderscited by 21
  20. 2025ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Modelscited by 17
  21. 2024Math Neurosurgery: Isolating Language Models' Math Reasoning Abilities Using Only Forward Passescited by 10
  22. 2025PEIRCE: Unifying Material and Formal Reasoning via LLM-Driven Neuro-Symbolic Refinementcited by 7
  23. 2025Self-Regularization with Sparse Autoencoders for Controllable LLM-based Classificationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 9.

13 papers.

Sort:

By area:

  1. 2026Purified OPSD: On-Policy Self-Distillation Without Losing How to Thinkcited by 31
  2. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  3. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
  4. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 18
  5. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 12
  6. 2025SGIC: A Self-Guided Iterative Calibration Framework for RAGcited by 8
  7. 2025Merge-of-Thought Distillationcited by 7
  8. 2025Prototype-Based Dynamic Steering for Large Language Modelscited by 6
  9. 2026GeoSteer: Faithful Chain-of-Thought Steering via Latent Manifold Gradientscited by 4
  10. 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Modelscited by 2
  11. 2026How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoningcited by 1
  12. 2026Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoderscited by 0
1 more
  1. 2026SelectTTRL: Representation-Guided Data Selection for Efficient Test-Time Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Activation Steering for Chain-of-Thought Compression
  2. 2025Understanding Reasoning in Thinking Language Models via Steering Vectors
  3. 2025Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization
  4. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding
  5. 2024Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding
  6. 2025Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information
  7. 2026Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
  8. 2024Chain-of-Thought Augmentation with Logit Contrast for Enhanced Reasoning in Language Models