mlsanityData to 7 Oct 2026

Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning

arXiv 2607.15736 ↗First posted 17 Jul 2026cs.CLcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large reasoning models often solve problems through long chain-of-thought (CoT) traces, yet much of this computation is spent on redundant derivations, repeated self-verification, and detours that do not improve the final answer. Existing on-policy self-distillation methods reduce this cost by matching a student model to a concise copy of itself on prefixes sampled from the student's own rollouts. We show that this objective has an initialization bottleneck. Since supervision is applied only to visited prefixes, training from a verbose base model places the KL loss on contexts that are often noisy, redundant, or already off track. In such regions, a concise teacher can provide only local corrections, while the student continues to explore trajectories that an efficient reasoner should avoid. In this paper, we propose BIRD(Bootstrapped Iterative Self-Reasoning Distillation), a two-stage self-reasoning distillation method that improves the rollout distribution before on-policy training. BIRD first samples concise solutions from the base model under a brevity instruction, keeps only answer-correct traces, and performs a lightweight prompt-switch SFT step. The traces are generated with the brevity instruction but learned under the original task prompt, turning instruction-induced conciseness into a default reasoning behavior. Starting from this warm model, BIRD then applies on-policy reverse-KL distillation with a concise self-teacher, now on cleaner and more informative prefixes. Across Qwen3 series models, BIRD achieves a stronger accuracy-efficiency trade-off than prompting and cold-start on-policy distillation on MATH-500 and AIME benchmarks. On Qwen3-8B, it improves MATH-500 accuracy from 86.2% to 92.0% while reducing the average response length from 3,099 to 1,115 tokens. These results highlight prefix support as a central factor in efficient reasoning distillation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 35 of its 38 references that are in the map. First posted on arXiv 17 Jul 2026.

Builds on

35 of 38 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation1 Jun
  2. 2026ROSD: Reflective On-Policy Self-Distillation for Language Model Reasoning across Domains27 May
  3. 2026CRISP: Compressed Reasoning via Iterative Self-Policy Distillation5 Mar
  4. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models26 Jan
  5. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models11 Jan
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 29

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  4. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  5. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  6. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  7. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 246
  8. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  9. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 167
  10. 2026TIP: Token Importance in On-Policy Distillationcited by 90
  11. 2025Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropycited by 37
  12. 2026OPSDL: On-Policy Self-Distillation for Long-Context Language Modelscited by 29
17 more
  1. 2024Over-Reasoning and Redundant Calculation of Large Language Modelscited by 27
  2. 2026Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoringcited by 21
  3. 2026PointCoT: A Multi-modal Benchmark for Explicit 3D Geometric Reasoningcited by 19
  4. 2025Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubtcited by 17
  5. 2025Leash: Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Modelcited by 13
  6. 2026Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinementcited by 13
  7. 2025Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoningcited by 11
  8. 2025Do LLMs Really Need 10+ Thoughts for "Find the Time 1000 Days Later"? Towards Structural Understanding of LLM Overthinkingcited by 10
  9. 2026PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learningcited by 10
  10. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Modelscited by 7
  11. 2026CtrlCoT: Dual-Granularity Chain-of-Thought Compression for Controllable Reasoningcited by 6
  12. 2026Mitigating Overthinking in Large Reasoning Models via Difficulty-aware Reinforcement Learningcited by 5
  13. 2026Intern-Atlas: A Methodological Evolution Graph as Research Infrastructure for AI Scientistscited by 5
  14. 2026Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoningcited by 3
  15. 2026CFMS: A Coarse-to-Fine Multimodal Synthesis Framework for Enhanced Tabular Reasoningcited by 2
  16. 2026FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Modelscited by 2
  17. 2026SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineeringcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models
  2. 2026Reasoning Compression with Mixed-Policy Distillation
  3. 2026Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning
  4. 2026Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
  5. 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
  6. 2026Learning from Partial Chain-of-Thought via Truncated-Reasoning Self-Distillation
  7. 2026Geometric Self-Distillation for Reasoning Generalization
  8. 2026Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning