# How Does Prefix Matter in Reasoning Model Tuning?

arXiv: https://arxiv.org/abs/2601.01624 · HTML: https://mlsanity.com/paper/2601.01624/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 4 Jan 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 37)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [A StrongREJECT for Empty Jailbreaks](https://mlsanity.com/paper/2402.10260/) (2024)
- [WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models](https://mlsanity.com/paper/2406.18510/) (2024)
- [SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities](https://mlsanity.com/paper/2502.12025/) (2025)
- [The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1](https://mlsanity.com/paper/2502.12659/) (2025)
- [STAR-1: Safer Alignment of Reasoning LLMs with 1K Data](https://mlsanity.com/paper/2504.01903/) (2025)
- [Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling](https://mlsanity.com/paper/2507.01679/) (2025)
- [RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability](https://mlsanity.com/paper/2504.10081/) (2025)
- [The Super Weight in Large Language Models](https://mlsanity.com/paper/2411.07191/) (2024)
- [Fine-tuning Large Language Models for Domain-specific Machine Translation](https://mlsanity.com/paper/2402.15061/) (2024)
- [The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models](https://mlsanity.com/paper/2503.02875/) (2025)
- [ToDi: Token-wise Distillation via Fine-Grained Divergence Control](https://mlsanity.com/paper/2505.16297/) (2025)
- [SaRO: Enhancing LLM Safety through Reasoning-based Alignment](https://mlsanity.com/paper/2504.09420/) (2025)
- [WildVis: Open Source Visualizer for Million-Scale Chat Logs in the Wild](https://mlsanity.com/paper/2409.03753/) (2024)
- [Uncovering Logit Suppression Vulnerabilities in LLM Safety Alignment](https://mlsanity.com/paper/2405.13068/) (2024)
- [Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace](https://mlsanity.com/paper/2503.01419/) (2025)
- [LookAhead Tuning: Safer Language Models via Partial Answer Previews](https://mlsanity.com/paper/2503.19041/) (2025)
- [UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases](https://mlsanity.com/paper/2507.21652/) (2025)
- [Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Is](https://mlsanity.com/paper/2507.21820/) (2025)
- [Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training](https://mlsanity.com/paper/2508.15390/) (2025)

## Abstract (arXiv)

Recent alignment studies commonly remove introductory boilerplate phrases from supervised fine-tuning (SFT) datasets. This work challenges that assumption. We hypothesize that safety- and reasoning-oriented prefix sentences serve as lightweight alignment signals that can guide model decoding toward safer and more coherent responses. To examine this, we fine-tune three R1 series models across three core model capabilities: reasoning (mathematics, coding), safety, and factuality, systematically varying prefix inclusion from 0% to 100%. Results show that prefix-conditioned SFT improves both safety and reasoning performance, yielding up to +6% higher Safe@1 accuracy on adversarial benchmarks (WildJailbreak, StrongReject) and +7% improvement on GSM8K reasoning. However, factuality and coding tasks show marginal or negative effects, indicating that prefix-induced narrowing of the search space benefits structured reasoning. Token-level loss analysis further reveals that prefix tokens such as "revised" and "logically" incur higher gradient magnitudes, acting as alignment anchors that stabilize reasoning trajectories. Our findings suggest that prefix conditioning offers a scalable and interpretable mechanism for improving reasoning safety, serving as an implicit form of alignment that complements traditional reward-based methods.