mlsanityData to 14 Oct 2026

On-Policy Context Distillation for Language Models

arXiv 2602.12275 ↗First posted 12 Feb 2026cs.CLcited by 186

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

Context distillation enables language models to internalize in-context knowledge into their parameters. In our work, we propose On-Policy Context Distillation (OPCD), a framework that bridges on-policy distillation with context distillation by training a student model on its own generated trajectories while minimizing reverse Kullback-Leibler divergence against a context-conditioned teacher. We demonstrate the effectiveness of OPCD on two important applications: experiential knowledge distillation, where models extract and consolidate transferable knowledge from their historical solution traces, and system prompt distillation, where models internalize beneficial behaviors encoded in optimized prompts. Across mathematical reasoning, text-based games, and domain-specific tasks, OPCD consistently outperforms baseline methods, achieving higher task accuracy while better preserving out-of-distribution capabilities. We further show that OPCD enables effective cross-size distillation, where smaller student models can internalize experiential knowledge from larger teachers.

Key facts. Cited by 186 papers in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 21 of its 23 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

21 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models26 Jan

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  6. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  7. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  8. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  9. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  10. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  11. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 417
  12. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 216
8 more
  1. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 176
  2. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 139
  3. 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 94
  4. 2020ETHOS: an Online Hate Speech Detection Datasetcited by 66
  5. 2025Black-Box On-Policy Distillation of Large Language Modelscited by 59
  6. 2025System Prompt Optimization with Meta-Learningcited by 18
  7. 2025InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformationcited by 17
  8. 2025Cogito, Ergo Ludo: An Agent that Learns to Play by Reasoning and Planningcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 186.

186 papers.

Sort:

By area:

  1. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 299
  2. 2026A Survey of On-Policy Distillation for Large Language Modelscited by 163
  3. 2026Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixescited by 145
  4. 2026Self-Distilled RLVRcited by 131
  5. 2026Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?cited by 103
  6. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agentscited by 90
  7. 2026Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routingcited by 76
  8. 2026CRISP: Compressed Reasoning via Iterative Self-Policy Distillationcited by 58
  9. 2026Self-Distilled Agentic Reinforcement Learningcited by 55
  10. 2026Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillationcited by 51
  11. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipecited by 49
  12. 2026TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agentscited by 47
38 more
  1. 2026Embarrassingly Simple Self-Distillation Improves Code Generationcited by 40
  2. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillationcited by 32
  3. 2026OPID: On-Policy Skill Distillation for Agentic Reinforcement Learningcited by 32
  4. 2026OPSDL: On-Policy Self-Distillation for Long-Context Language Modelscited by 30
  5. 2026PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competencecited by 28
  6. 2026Online Experiential Learning for Language Modelscited by 28
  7. 2026AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillationcited by 26
  8. 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Whycited by 25
  9. 2026D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Modelscited by 22
  10. 2026SOD: Step-wise On-policy Distillation for Small Language Model Agentscited by 21
  11. 2026GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillationcited by 19
  12. 2026MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debatecited by 17
  13. 2026On-Policy Self-Distillation without Any Supervisioncited by 17
  14. 2026Trajectory-Refined Distillationcited by 16
  15. 2026Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVRcited by 15
  16. 2026Multi-Turn On-Policy Distillation with Prefix Replaycited by 15
  17. 2026Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Informationcited by 14
  18. 2026Rethinking Reward Supervision: Rubric-Conditioned Self-Distillationcited by 13
  19. 2026SimCT: Recovering Lost Supervision for Cross-Tokenizer On-Policy Distillationcited by 12
  20. 2026ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillationcited by 12
  21. 2026DOPD: Dual On-policy Distillationcited by 12
  22. 2026AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learningcited by 12
  23. 2026Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillationcited by 12
  24. 2026Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Trainingcited by 11
  25. 2026OPRD: On-Policy Representation Distillationcited by 11
  26. 2026HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillationcited by 11
  27. 2026ATOD: Annealed Turn-Aware On-Policy Distillation for Multi-Turn Agentic Taskscited by 11
  28. 2026Multi-Rollout On-Policy Distillation via Peer Successes and Failurescited by 10
  29. 2026Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoningcited by 10
  30. 2026Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillationcited by 10
  31. 2026On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversitycited by 10
  32. 2026The Illusion of Certainty: Decoupling Capability and Calibration in On-Policy Distillationcited by 9
  33. 2026From Generic Correlation to Input-Specific Credit in On-Policy Self Distillationcited by 9
  34. 2026Rethinking Privileged Information in On-Policy Self-Distillationcited by 9
  35. 2026Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillationcited by 8
  36. 2026Self-Distilled Policy Gradientcited by 8
  37. 2026UI-MOPD: Multi-Platform On-Policy Distillation for Unified GUI Agentscited by 8
  38. 2026Weak-to-Strong On-Policy Distillationcited by 8

50 most cited of 186 All 186 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes
  2. 2026A Brief Overview: On-Policy Self-Distillation In Large Language Models
  3. 2026Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
  4. 2026OVD: On-policy Verbal Distillation
  5. 2026Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
  6. 2026Teacher-Guided Policy Optimization for On-Policy Reasoning Distillation under Large Policy Divergence
  7. 2026Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress
  8. 2026EasyOPD: An Easy-to-use On-Policy Distillation Framework for Large Language Models