mlsanityData to 14 Oct 2026

Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models

arXiv 2608.16647 ↗First posted 17 Aug 2026cs.CLcited by 4

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

On-policy distillation (OPD) transfers teacher capabilities by supervising trajectories sampled from the student's own policy, yet its generalization behavior remains poorly understood, as most studies evaluate OPD on a single domain and on benchmarks close to the training data. We present a controlled study that varies one generalization factor at a time, from in-domain distribution shifts to cross-domain transfer and the multi-teacher setting. We find that OPD transfers a teacher's reasoning behavior rather than its answers to particular problems: training difficulty barely matters, and even problems the teacher never solves are useful. Transfer depends strongly on the origin relationship between teacher and student: same-origin pairs bring the student close to the teacher across languages, reasoning horizons, and even other domains, whereas cross-origin pairs mostly fit the trained distribution. This broad reach is a double-edged sword: since routing prompts to domain experts cannot confine each teacher's influence, combining them yields a mixture-dependent seesaw among their capabilities. These results clarify when OPD generalizes and offer a useful perspective for diagnosing multi-teacher OPD.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 48 of its 58 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

48 of 58 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations15 Jul
  2. 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training29 Jun
  3. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation1 Jun
  4. 2026Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation26 May
  5. 2026On-Policy Distillation with Best-of-N Teacher Rollout Selection10 May
  6. 2026Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level7 May
  7. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe5 May
  8. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe14 Apr
  9. 2026SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting12 Apr
  10. 2026Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Models9 Apr
  11. 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes26 Mar
  12. 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation19 Mar

Other references · 36

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  9. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  10. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  11. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  12. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
24 more
  1. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  2. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  3. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  4. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 194
  5. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  6. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 174
  7. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  8. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 158
  9. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 109
  10. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 70
  11. 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
  12. 2025JustRL: Scaling a 1.5B LLM with a Simple RL Recipecited by 64
  13. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 59
  14. 2025SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domaincited by 31
  15. 2025Shape of Thought: When Distribution Matters More than Correctness in Reasoning Taskscited by 11
  16. 2025Rethinking the Sampling Criteria in Reinforcement Learning for LLM Reasoning: A Competence-Difficulty Alignment Perspectivecited by 9
  17. 2025LearnAlign: Data Selection for LLM Reinforcement Learning with Improved Gradient Alignmentcited by 9
  18. 2025R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?cited by 8
  19. 2025Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5Bcited by 8
  20. 2026On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuningcited by 4
  21. 2026KAT-Coder-V2.5 Technical Reportcited by 4
  22. 2026Mach-Mind-4-Flash Technical Reportcited by 4
  23. 2025Exploring How LLMs Capture and Represent Domain-Specific Knowledgecited by 3
  24. 2026Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Data-Free On-Policy Distillation: How Far Can We Go Without External Data?cited by 1
  2. 2026Beyond Prompt Count: How Data Shapes Transfer in On-Policy Distillationcited by 0
  3. 2026Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shiftscited by 0
  4. 2026Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes
  2. 2026Geometric Self-Distillation for Reasoning Generalization
  3. 2026A Brief Overview: On-Policy Self-Distillation In Large Language Models
  4. 2026A Survey of On-Policy Distillation for Large Language Models
  5. 2026Reward-Aligned Reweighting for On-Policy Distillation
  6. 2026Slow-Fast Multi-Teacher On-Policy Distillation for Capability Preservation
  7. 2026Fast and Effective On-policy Distillation from Reasoning Prefixes
  8. 2026Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress