# OPSDL: On-Policy Self-Distillation for Long-Context Language Models

arXiv: https://arxiv.org/abs/2604.17535 · HTML: https://mlsanity.com/paper/2604.17535/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 17 of its 17 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Placed in

- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 17 of 17)

- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [YaRN: Efficient Context Window Extension of Large Language Models](https://mlsanity.com/paper/2309.00071/) (2023)
- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [Reinforcement Learning via Self-Distillation](https://mlsanity.com/paper/2601.20802/) (2026)
- [On-Policy Context Distillation for Language Models](https://mlsanity.com/paper/2602.12275/) (2026)
- [Qwen2.5-1M Technical Report](https://mlsanity.com/paper/2501.15383/) (2025)
- [QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning](https://mlsanity.com/paper/2505.17667/) (2025)
- [LongReward: Improving Long-context Large Language Models with AI Feedback](https://mlsanity.com/paper/2410.21252/) (2024)
- [QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management](https://mlsanity.com/paper/2512.12967/) (2025)
- [LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization](https://mlsanity.com/paper/2502.13922/) (2025)
- [Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs](https://mlsanity.com/paper/2509.21361/) (2025)
- [SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization](https://mlsanity.com/paper/2505.11166/) (2025)
- [MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling](https://mlsanity.com/paper/2602.11761/) (2026)

## Built on by (most cited first, 30 of 30)

- [Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation](https://mlsanity.com/paper/2605.26844/) (2026)
- [Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level](https://mlsanity.com/paper/2605.06387/) (2026)
- [Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation](https://mlsanity.com/paper/2606.02684/) (2026)
- [Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning](https://mlsanity.com/paper/2606.18974/) (2026)
- [Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding](https://mlsanity.com/paper/2605.00642/) (2026)
- [Crosslingual On-Policy Self-Distillation for Multilingual Reasoning](https://mlsanity.com/paper/2605.09548/) (2026)
- [Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning](https://mlsanity.com/paper/2605.22263/) (2026)
- [Latent On-Policy Self-Distillation](https://mlsanity.com/paper/2608.13040/) (2026)
- [MAIGO: Mitigating Lost-in-Conversation with History-Cleaned On-Policy Self-Distillation](https://mlsanity.com/paper/2605.27186/) (2026)
- [Context Distillation as Latent Memory Management](https://mlsanity.com/paper/2605.28889/) (2026)
- [Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation](https://mlsanity.com/paper/2608.09826/) (2026)
- [boldsymbolf-OPD: Stabilizing Long-Horizon On-Policy Distillation with Freshness-Aware Control](https://mlsanity.com/paper/2605.17862/) (2026)
- [LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks](https://mlsanity.com/paper/2607.18110/) (2026)
- [Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models](https://mlsanity.com/paper/2608.27065/) (2026)
- [Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models](https://mlsanity.com/paper/2605.30251/) (2026)
- [Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories](https://mlsanity.com/paper/2606.03979/) (2026)
- [Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning](https://mlsanity.com/paper/2607.15736/) (2026)
- [β-OPSD: Deriving with Policy Optimization, Training with Self-Distillation](https://mlsanity.com/paper/2607.28582/) (2026)
- [Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation](https://mlsanity.com/paper/2608.11191/) (2026)
- [Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation](https://mlsanity.com/paper/2609.23697/) (2026)
- [Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold](https://mlsanity.com/paper/2607.27770/) (2026)
- [TransMem: Transforming Hidden States into Memory for Large Language Models](https://mlsanity.com/paper/2607.29032/) (2026)
- [Bidirectional Context Self-Distillation for Reinforcement Learning of Skill-Based LLM Agents](https://mlsanity.com/paper/2608.09555/) (2026)
- [PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents](https://mlsanity.com/paper/2609.23449/) (2026)
- [Learn How to Act from Your Own Interactions: On-Policy Self-Distillation for GUI Agents](https://mlsanity.com/paper/2609.27307/) (2026)
- [Rubric-Aware On-Policy Self-Distillation for LLM Personalization](https://mlsanity.com/paper/2609.35262/) (2026)
- [d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models](https://mlsanity.com/paper/2609.35362/) (2026)
- [Learning When and How to Intervene: A Hindsight-Distilled Sentinel for Coding Agents](https://mlsanity.com/paper/2609.39957/) (2026)
- [Privileged Context as Drift in On-Policy Self-Distillation](https://mlsanity.com/paper/2610.07842/) (2026)
- [MIRROR: From Imitation to Internalization in LLM Personalization](https://mlsanity.com/paper/2610.09795/) (2026)

## Abstract (arXiv)

Extending the effective context length of large language models (LLMs) remains a central challenge for real-world applications. While recent post-training methods have made progress in long-context scaling, they either rely on high-quality supervision data or sparse sequence-level rewards, leading to unstable and inefficient optimization. We propose OPSDL, an On-Policy Self-Distillation method for enhancing the Long-context capabilities of LLMs. Unlike other recent self-distillation methods that inject privileged information and rely on the model's in-context learning ability to act as a teacher, OPSDL leverages the model's own inherently strong short-context capability as a self-teacher to supervise its own generation in long-context scenarios. The model first generates responses conditioned on the full long-context, then the self-teacher provides per-token supervision signals via point-wise reverse KL divergence under the relevant extracted short-context. This dense token-level signal encourages faithful use of relevant evidence and mitigates hallucinations induced by irrelevant context. We evaluate OPSDL on long-context benchmarks across a range of models from 7B to 32B parameters. Results show consistent and substantial improvements across varying context lengths, outperforming standard post-training approaches such as SFT and DPO with higher sample efficiency. Notably, these gains are achieved without degrading general short-context performance. These findings highlight the effectiveness of OPSDL as a scalable and stable approach for long-context learning.