mlsanityData to 14 Oct 2026

How to Fine-Tune a Reasoning Model? A Teacher-Student Cooperation Framework to Synthesize Student-Consistent SFT Data

arXiv 2604.14164 ↗First posted 23 Mar 2026cs.CLcited by 8

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

A widely adopted strategy for model enhancement is to use synthetic data generated by a stronger model for supervised fine-tuning (SFT). However, for emerging reasoning models like Qwen3-8B, this approach often fails to improve reasoning capabilities and can even lead to a substantial drop in performance. In this work, we identify substantial stylistic divergence between teacher generated data and the distribution of student as a major factor impacting SFT. To bridge this gap, we propose a Teacher-Student Cooperation Data Synthesis framework (TESSY), which interleaves teacher and student models to alternately generate style and non-style tokens. Consequently, TESSY produces synthetic sequences that inherit the advanced reasoning capabilities of the teacher while maintaining stylistic consistency with the distribution of the student. In experiments on code generation using GPT-OSS-120B as the teacher, fine-tuning Qwen3-8B on teacher-generated data leads to performance drops of 3.25% on LiveCodeBench-Pro and 10.02% on OJBench, whereas TESSY achieves improvements of 11.25% and 6.68%.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 50 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

21 of 50 references.

References · 21

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  7. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  8. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  9. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
  10. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 113
  11. 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 94
  12. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 79
9 more
  1. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 77
  2. 2025LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?cited by 55
  3. 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoningcited by 51
  4. 2025OJBench: A Competition Level Code Benchmark For Large Language Modelscited by 26
  5. 2024A Survey of Neural Code Intelligence: Paradigms, Advances and Beyondcited by 12
  6. 2025AdaSwitch: Balancing Exploration and Guidance in Knowledge Distillation via Adaptive Switchingcited by 6
  7. 2025StyleBench: Evaluating thinking styles in Large Language Modelscited by 5
  8. 2025Your thoughts tell who you are: Characterize the reasoning patterns of LRMscited by 4
  9. 2025Pipelined Decoder for Efficient Context-Aware Text Generationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026SocraticPO: Policy Optimization via Interactive Guidancecited by 4
  2. 2026When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoffcited by 4
  3. 2026Execution-First Synthetic Tool-Use Trace Generation for LLM Agentscited by 2
  4. 2026Shared SFT Lessons Across Alignment, Model Organisms, and Toy Modelscited by 1
  5. 2026Learning to Adapt SFT Data for Better Reasoning Generalizationcited by 0
  6. 2026Agentic Instruction Data Selection: Let DataMaster Interpret Your Intentcited by 0
  7. 2026Reason in Style: Discovering and Controlling Style in Language Modelscited by 0
  8. 2026Continually Post-training Small Models via Synthetic Self-distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Large Language Models Are Reasoning Teachers
  2. 2026Lightning OPD 2.0: Mitigating Style Bias in Cross-Teacher On-Policy Distillation for Large Reasoning Models
  3. 2026Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decoding
  4. 2025Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate
  5. 2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
  6. 2026Long-Chain Reasoning Distillation via Adaptive Prefix Alignment
  7. 2025The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
  8. 2026Collaborative Reasoning Distillation via Cross-Feedback and Coherent Curation