mlsanityData to 14 Oct 2026

Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation

arXiv 2609.02998 ↗First posted 2 Sep 2026cs.LG cs.AI cs.CLcited by 5

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

On-policy distillation (OPD) accelerates post-training by providing dense token-level supervision from a frozen teacher on the student's own rollouts. Vanilla OPD applies this supervision uniformly across prompts, without checking whether the teacher is reliable for each prompt. Because reverse KL is mode-seeking, a confidently wrong teacher can induce a strong yet misleading update. Distributional proxies, such as entropy or teacher-student likelihood agreement, measure uncertainty or agreement but do not directly verify outcome correctness. We introduce Teacher-Gated On-Policy Distillation (TGOPD), built on the principle that teacher reliability should be verified at the prompt level before dense supervision is admitted. TGOPD estimates reliability from a small set of verifier-scored teacher probes and routes each prompt exclusively to dense OPD when the reliability check passes or to verifier-grounded GRPO otherwise. Across 4B and 35B students in mathematics, code, and instruction following, TGOPD outperforms Vanilla OPD in all six single-domain settings and achieves higher seven-benchmark averages at both scales under multi-domain training. By using otherwise-idle teacher capacity for reliability estimation, TGOPD also reduces teacher-side compute waste in asynchronous OPD, increasing teacher-node GPU utilization from 9.8% to 78.9% in the measured 4B single-domain run.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 24 of its 28 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

24 of 28 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026REOPD: Reliability-Adaptive Reward Extrapolation for On-Policy Distillation12 Aug
  2. 2026SPOT: Sparse Probing and Outcome Calibration for On-Policy Distillation5 Aug
  3. 2026Reward-Gated On-Policy Distillation4 Jul
  4. 2026MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training29 Jun
  5. 2026Trust Region On-Policy Distillation31 May
  6. 2026When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning20 May
  7. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe14 Apr
  8. 2026Self-Distilled RLVR3 Apr
  9. 2026A Survey of On-Policy Distillation for Large Language Models1 Apr
  10. 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes26 Mar
  11. 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation19 Mar
  12. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillation11 Mar
2 more
  1. 2026Entropy-Aware On-Policy Distillation of Language Models7 Mar
  2. 2026GATES: Self-Distillation under Privileged Context with Consensus Gating24 Feb

Other references · 10

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  4. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  5. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  6. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  7. 2025Small Models Struggle to Learn from Strong Reasonerscited by 113
  8. 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 90
  9. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 55
  10. 2025Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Modelcited by 55

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learningcited by 3
  2. 2026Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillationcited by 2
  3. 2026Trajectory Learnability for Offline On-Policy Distillation with Imperfect Teacherscited by 0
  4. 2026DuoOPD: Learning from Joint Teacher-Student Outcomes for Multi-Task On-Policy Distillationcited by 0
  5. 2026Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning Teacherscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ReOrder-OPD:Reliability-Aware Prompt Ordering for On-Policy Distillation
  2. 2026Correcting or Continuing? What On-Policy Distillation Learns from Student Errors
  3. 2026When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
  4. 2026SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision
  5. 2026SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling
  6. 2026Calibrating Teacher--Student Discrepancy for On-Policy Distillation
  7. 2026The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes
  8. 2026Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?