mlsanityData to 14 Oct 2026

Steering LLM Reasoning Through Bias-Only Adaptation

arXiv 2505.18706 ↗First posted 24 May 2025cs.LG cs.AIcited by 16

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

We show that training a single d-dimensional steering vector per layer with reinforcement learning, while freezing all base weights, matches the accuracy of fully RL-tuned reasoning models on mathematical-reasoning tasks. On an 8 billion-parameter model this adds only ≈ 0.0016% additional parameters and reproduces performance across a range of base models and mathematical-reasoning benchmarks. These results tighten the upper bound on the parameter budget required for high-level chain-of-thought reasoning, indicating that millions of adapter weights are unnecessary. The minimal trainable footprint reduces optimizer memory and inter-GPU communication, lowering the overall cost of fine-tuning. Moreover, a logit-lens analysis shows that the learned vectors amplify coherent token directions, providing clearer insight into the model's internal computations.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 23 of its 36 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

23 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Rethinking Reflection in Pre-Training5 Apr

Other references · 22

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  11. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  12. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
10 more
  1. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  2. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  3. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  4. 2023Steering Language Models With Activation Engineeringcited by 922
  5. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  6. 2025LIMO: Less is More for Reasoningcited by 498
  7. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  8. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  9. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 118
  10. 2025Tell me about yourself: LLMs are aware of their learned behaviorscited by 91

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 12.

16 papers.

Sort:

By area:

  1. 2025Test-Time Adaptation for LLM Agents via Environment Interactioncited by 20
  2. 2025Small Vectors, Big Effects: A Mechanistic Study of RL-Induced Reasoning via Steering Vectorscited by 8
  3. 2026Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMscited by 7
  4. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
  5. 2026Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lenscited by 6
  6. 2026Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistencycited by 5
  7. 2026KV-Skill: Forging Expertise in the Model's Native Languagecited by 4
  8. 2026Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Modelscited by 2
  9. 2025BEFT: Bias-Efficient Fine-Tuning of Language Models in Low-Data Regimescited by 1
  10. 2026Transcoder Adapters for Reasoning-Model Diffingcited by 1
  11. 2026Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimizationcited by 1
  12. 2026When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Searchcited by 1
4 more
  1. 2026Label-free steering: Compressing test-time reinforcement learning into bias-only subspacescited by 1
  2. 2025Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustnesscited by 0
  3. 2026Inference-Time Policy Alignment for Fair Reinforcement Learningcited by 0
  4. 2026Attention is Case-Sensitivecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Ponder: Adaptive Reasoning in Latent Space
  2. 2025Unleashing the Reasoning Potential of Pre-trained LLMs by Critique Fine-Tuning on One Problem
  3. 2025Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control
  4. 2025Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
  5. 2025RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning
  6. 2025SABER: Switchable and Balanced Training for Efficient LLM Reasoning
  7. 2025VAR-MATH: Probing True Mathematical Reasoning in LLMS via Symbolic Multi-Instance Benchmarks
  8. 2026RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering