SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision
Not on the map. Why?
Abstract (arXiv)
On-policy distillation (OPD) has a property absent in offline distillation and RL: teacher supervision quality depends on student competence. Incoherent rollouts yield noisy gradients; already-mastered tokens yield redundant ones. This creates waste at three scales (tokens, training phases, and prompts) yet existing methods supervise uniformly. We introduce SEAD, which uses entropy as a unified probe of this competence-dependent degradation at three scales: (1) joint teacher-student entropy partitions tokens into zones receiving tailored divergences or zero gradient (approx. 50% skipped); (2) a cosine schedule anneals from forward to reverse KL as competence grows; (3) a competence-gated curriculum introduces prompts easy-to-hard. These components are symbiotically necessary: token selection requires coherent rollouts (curriculum), annealing requires monotonic improvement (also curriculum). On OLMo-3 (7B to 32B), SEAD achieves +4.8 avg accuracy over vanilla OPD across six math benchmarks, with ablations confirming super-additive interactions.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 19 references that are in the map. First posted on arXiv 26 Jun 2026.
Builds on
18 of 19 references.
References · 18
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
- 2015Policy Distillationcited by 414
- 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
- 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 262
- 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 201
- 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
- 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 157
- 2026Entropy-Aware On-Policy Distillation of Language Modelscited by 151
- 2026A Survey of On-Policy Distillation for Large Language Modelscited by 148
6 more
- 2026TIP: Token Importance in On-Policy Distillationcited by 90
- 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 89
- 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillationcited by 71
- 2026Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationcited by 28
- 2026Rethinking Selective Knowledge Distillationcited by 6
- 2026Diversity-Aware Reverse Kullback-Leibler Divergence for Large Language Model Distillationcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.