# Steering LLM Reasoning Through Bias-Only Adaptation

arXiv: https://arxiv.org/abs/2505.18706 · HTML: https://mlsanity.com/paper/2505.18706/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 24 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 23 of 23)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://mlsanity.com/paper/2503.18892/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://mlsanity.com/paper/2503.24290/) (2025)
- [Reinforcement Learning for Reasoning in Large Language Models with One Training Example](https://mlsanity.com/paper/2504.20571/) (2025)
- [Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization](https://mlsanity.com/paper/2406.00045/) (2024)
- [Tell me about yourself: LLMs are aware of their learned behaviors](https://mlsanity.com/paper/2501.11120/) (2025)
- [Rethinking Reflection in Pre-Training](https://mlsanity.com/paper/2504.04022/) (2025)

## Built on by (most cited first, 16 of 16)

- [Test-Time Adaptation for LLM Agents via Environment Interaction](https://mlsanity.com/paper/2511.04847/) (2025)
- [Small Vectors, Big Effects: A Mechanistic Study of RL-Induced Reasoning via Steering Vectors](https://mlsanity.com/paper/2509.06608/) (2025)
- [Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs](https://mlsanity.com/paper/2602.07276/) (2026)
- [Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures](https://mlsanity.com/paper/2601.19928/) (2026)
- [Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens](https://mlsanity.com/paper/2604.02608/) (2026)
- [Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency](https://mlsanity.com/paper/2603.12298/) (2026)
- [KV-Skill: Forging Expertise in the Model's Native Language](https://mlsanity.com/paper/2608.05475/) (2026)
- [Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models](https://mlsanity.com/paper/2603.14636/) (2026)
- [BEFT: Bias-Efficient Fine-Tuning of Language Models in Low-Data Regimes](https://mlsanity.com/paper/2509.15974/) (2025)
- [Transcoder Adapters for Reasoning-Model Diffing](https://mlsanity.com/paper/2602.20904/) (2026)
- [Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization](https://mlsanity.com/paper/2604.04231/) (2026)
- [When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search](https://mlsanity.com/paper/2605.16362/) (2026)
- [Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces](https://mlsanity.com/paper/2609.18587/) (2026)
- [Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness](https://mlsanity.com/paper/2506.24056/) (2025)
- [Inference-Time Policy Alignment for Fair Reinforcement Learning](https://mlsanity.com/paper/2608.00175/) (2026)
- [Attention is Case-Sensitive](https://mlsanity.com/paper/2608.03711/) (2026)

## Abstract (arXiv)

We show that training a single d-dimensional steering vector per layer with reinforcement learning, while freezing all base weights, matches the accuracy of fully RL-tuned reasoning models on mathematical-reasoning tasks. On an 8 billion-parameter model this adds only ≈ 0.0016% additional parameters and reproduces performance across a range of base models and mathematical-reasoning benchmarks. These results tighten the upper bound on the parameter budget required for high-level chain-of-thought reasoning, indicating that millions of adapter weights are unnecessary. The minimal trainable footprint reduces optimizer memory and inter-GPU communication, lowering the overall cost of fine-tuning. Moreover, a logit-lens analysis shows that the learned vectors amplify coherent token directions, providing clearer insight into the model's internal computations.