# On-policy and self-distillation for LLMs

Area: [Knowledge distillation](https://mlsanity.com/distill/index.md). HTML: https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/

## Key facts

- On-policy and self-distillation for LLMs: 446 papers on arXiv since 2021 (data through 14 Oct 2026).
- 167 of its 446 mapped papers (37%) cite “Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models” (2026, arXiv 2601.18734).
- 156 of its 446 mapped papers (35%) cite “Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe” (2026, arXiv 2604.13016).
- 184 new papers 25 Sep–14 Oct.

## Most cited here

- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026), cited by 167 papers here
- [Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe](https://mlsanity.com/paper/2604.13016/) (2026), cited by 156 papers here
- [Reinforcement Learning via Self-Distillation](https://mlsanity.com/paper/2601.20802/) (2026), cited by 128 papers here
- [Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation](https://mlsanity.com/paper/2602.12125/) (2026), cited by 103 papers here
- [Entropy-Aware On-Policy Distillation of Language Models](https://mlsanity.com/paper/2603.07079/) (2026), cited by 103 papers here
- [Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes](https://mlsanity.com/paper/2603.25562/) (2026), cited by 90 papers here
- [A Survey of On-Policy Distillation for Large Language Models](https://mlsanity.com/paper/2604.00626/) (2026), cited by 90 papers here
- [Self-Distillation Enables Continual Learning](https://mlsanity.com/paper/2601.19897/) (2026), cited by 89 papers here
- [On-Policy Context Distillation for Language Models](https://mlsanity.com/paper/2602.12275/) (2026), cited by 88 papers here
- [Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?](https://mlsanity.com/paper/2603.24472/) (2026), cited by 64 papers here

## New (25 Sep–14 Oct)

- [Do As I Do, Not As I Say: On-Policy Distillation Inherits the Teacher in Vision-Language Models](https://mlsanity.com/paper/2610.20989/) (2026-10-14)
- [Recovering Reliable Supervision via Critique-Guided On-Policy Distillation](https://mlsanity.com/paper/2610.20572/) (2026-10-14)
- [VASPO: Self-Distillation with Privileged Value Estimation](https://mlsanity.com/paper/2610.20516/) (2026-10-14)
- [RM-OPD: On-Policy Distillation for Generative Reward Models](https://mlsanity.com/paper/2610.20174/) (2026-10-14)
- [ActionOPD: Distilling Decisions, Not Tokens](https://mlsanity.com/paper/2610.19820/) (2026-10-14)
- [More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning](https://mlsanity.com/paper/2610.19789/) (2026-10-14)
- [Function-Space Temporal Self-Distillation for Policy Optimization](https://mlsanity.com/paper/2610.19687/) (2026-10-14)
- [Divergence-Routed On-Policy Distillation for Language Models](https://mlsanity.com/paper/2610.19572/) (2026-10-14)
- [DTDD: Divergence-Triggered Dynamic Distillation for Reliable On-Policy Supervision](https://mlsanity.com/paper/2610.19302/) (2026-10-13)
- [Outcome-Routed Distillation and Reinforcement Learning for Diffusion Language Models](https://mlsanity.com/paper/2610.19271/) (2026-10-13)
- [Demystifying On-Policy Distillation: Where Does the Teacher's Advantage Live?](https://mlsanity.com/paper/2610.19145/) (2026-10-13)
- [Does self-distillation improve reasoning?](https://mlsanity.com/paper/2610.18613/) (2026-10-13)
- [Imprinting: Knowledge Distillation Can Shift Student Behavior](https://mlsanity.com/paper/2610.18517/) (2026-10-13)
- [Teacher-Guided Tool Learning for Small Language Models via Bidirectional KL Constraints](https://mlsanity.com/paper/2610.18459/) (2026-10-13)
- [Teach Me After You Learn It: Lookahead Self-Distillation for RLVR](https://mlsanity.com/paper/2610.18256/) (2026-10-13)
- [LoOPSD: On-Policy Self-Distillation with Looped Teachers](https://mlsanity.com/paper/2610.18162/) (2026-10-13)
- [Failure Recovery via Belief Updates and World Modeling for Long-Horizon Agents](https://mlsanity.com/paper/2610.18082/) (2026-10-13)
- [Correcting or Continuing? What On-Policy Distillation Learns from Student Errors](https://mlsanity.com/paper/2610.18011/) (2026-10-13)
- [Explore with Memory, Learn without It: In-Context Exploration and Self-Distillation for LLM Reasoning](https://mlsanity.com/paper/2610.17734/) (2026-10-13)
- [RELAY: Selective Self-Distillation with Adaptive Hand-off for Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2610.17623/) (2026-10-13)
- [Training Generalist Navigation Policies via Action Self-Distillation](https://mlsanity.com/paper/2610.17593/) (2026-10-13)
- [Self-Distillation with Verbal Regularization](https://mlsanity.com/paper/2610.16354/) (2026-10-12)
- [Radioactive Feedback: Tracing Models Trained by Label-Only LLM Supervision](https://mlsanity.com/paper/2610.15685/) (2026-10-11)
- [A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoning](https://mlsanity.com/paper/2610.13744/) (2026-10-09)
- [Defence against the dark arts of distillation: Unsupervised discovery of subliminal learning](https://mlsanity.com/paper/2610.13536/) (2026-10-09)
- [Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2610.13521/) (2026-10-09)
- [Branching On-Policy Distillation](https://mlsanity.com/paper/2610.13481/) (2026-10-09)
- [CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation](https://mlsanity.com/paper/2610.13257/) (2026-10-09)
- [State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents](https://mlsanity.com/paper/2610.13238/) (2026-10-09)
- [Rethinking Relative Policy Differences Transfer of On-Policy Distillation](https://mlsanity.com/paper/2610.13205/) (2026-10-09)