mlsanityData to 7 Oct 2026

Small Vectors, Big Effects: A Mechanistic Study of RL-Induced Reasoning via Steering Vectors

arXiv 2509.06608 ↗First posted 8 Sep 2025cs.LGcited by 8

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

The mechanisms by which reasoning training reshapes LLMs' internal computations remain unclear. We study lightweight steering vectors inserted into the base model's residual stream and trained with a reinforcement-learning objective. These vectors explain a large portion of full fine-tuning performance increase while preserving the interpretability of small, additive interventions. We find that (i) the last-layer steering vector acts like a token-substitution bias concentrated on the first generated token, consistently boosting tokens such as "To" and "Step"; (ii) the penultimate-layer vector leaves attention patterns largely intact and instead operates through the MLP and unembedding, preferentially up-weighting process words and structure symbols; and (iii) the steering vectors transfer to other models from the same family. Taken together, these results deepen understanding of how trained steering vectors shape computation and should inform future work in activation engineering and the study of reasoning models.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 24 of its 36 references that are in the map. First posted on arXiv 8 Sep 2025.

Builds on

24 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Reasoning-Finetuning Repurposes Latent Representations in Base Models16 Jul
  2. 2025Understanding Reasoning in Thinking Language Models via Steering Vectors22 Jun
  3. 2025Steering LLM Reasoning Through Bias-Only Adaptation24 May

Other references · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  6. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  7. 2024OpenAI o1 System Cardcited by 1,220
  8. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  11. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  12. 2023Steering Language Models With Activation Engineeringcited by 824
9 more
  1. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  2. 2025LIMO: Less is More for Reasoningcited by 488
  3. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  4. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
  5. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  6. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  7. 2024Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMscited by 103
  8. 2025Tell me about yourself: LLMs are aware of their learned behaviorscited by 86
  9. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 72

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026From Weights to Activations: Is Steering the Next Frontier of Adaptation?cited by 8
  2. 2026What Drives Representation Steering? A Mechanistic Case Study on Steering Refusalcited by 7
  3. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 5
  4. 2026KV-Skill: Forging Expertise in the Model's Native Languagecited by 3
  5. 2026Reliable Control-Point Selection for Steering Reasoning in Large Language Modelscited by 1
  6. 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoningcited by 1
  7. 2025Sparse Concept Anchoring for Interpretable and Controllable Neural Representationscited by 0
  8. 2026Activation Flow: Manufacturing Activations for Steeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness
  2. 2025Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
  3. 2026The Halt Vector: Internalizing a Causal Steering Intervention for Efficient Reasoning
  4. 2026RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering
  5. 2026Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
  6. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
  7. 2025Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning
  8. 2025Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time