mlsanityData to 14 Oct 2026

JustOPD: On-Policy Distillation from Autoregressive to Diffusion Language Models

arXiv 2610.17397 ↗First posted 12 Oct 2026cs.LGcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

On-policy distillation (OPD) is a post-training approach for large language models (LLMs) that enables a student model to learn a teacher model's distribution from sequences sampled from the student's own policy. Despite its promising capability, OPD for diffusion language models (DLMs) remains underexplored and existing studies primarily focuses on DLM-to-DLM distillation. The limited availability of strong teacher DLMs and the flexible generation order of DLMs impose additional technical challenges as well. However, it remains unclear how to effectively distill knowledge from a strong autoregressive language model (ARM) into a weaker DLM, given the mismatch between the left-to-right decoding of ARMs and the flexible decoding order of DLMs. Inspired by Ni et al. (2026), we propose JustOPD, an ARM-to-DLM on-policy distillation method that uses autoregressive rollouts during training to align the student's generation process with the ARM teacher's left-to-right decoding. This alignment enables effective distillation from ARM teachers while preserving the DLM's flexible generation-order capability at inference time. Experiments using Qwen3-14B as the teacher and Dream-7B as the student demonstrate that JustOPD consistently improves mathematical reasoning performance over the base DLM across multiple divergence choices.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 38 of its 46 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

38 of 46 references.

Closest ancestors · 17

Same direction, newest first

  1. 2026Hesitation-Aware On-Policy Distillation for Diffusion Language Models27 Sep
  2. 2026ForkLeft: Entropy-First Rollouts for Prefix-Aligned Autoregressive-to-Diffusion Distillation26 Sep
  3. 2026Trace-Based On-Policy Distillation for Masked Diffusion Language Models18 Jul
  4. 2026dOPSD: On-Policy Self-Distillation for Diffusion Language Models5 Jul
  5. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation4 Jun
  6. 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models21 Jan
  7. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed16 Dec
  8. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  9. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning2 Oct
  10. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models8 Sep
  11. 2025Dream 7B: Diffusion Large Language Models21 Aug
  12. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
5 more
  1. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  2. 2025Large Language Diffusion Models14 Feb
  3. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  5. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun

Other references · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  6. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 455
  7. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  8. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 299
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  10. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  11. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 184
  12. 2026Entropy-Aware On-Policy Distillation of Language Modelscited by 175
9 more
  1. 2026A Survey of On-Policy Distillation for Large Language Modelscited by 163
  2. 2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 34
  3. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 24
  4. 2026Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvementcited by 11
  5. 2026OPD+: Rethinking the Advantage Design for On-Policy Distillationcited by 8
  6. 2026Step-Level On-Policy Distillation: Interpolating Between On-Policy Distillation and Supervised Fine-Tuningcited by 8
  7. 2026Learning from the Self-future: On-policy Self-distillation for dLLMscited by 5
  8. 2026A Continuous-Time Reinforcement Learning Framework for Fine-Tuning Discrete Diffusion Modelscited by 2
  9. 2026Understanding On-Policy Distillation: A Mechanistic Interpretability Perspective via Sparse Crosscoderscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales
  2. 2026d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models
  3. 2026Towards Unbiased On-Policy Distillation for Block Diffusion Language Models
  4. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
  5. 2026Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families
  6. 2023MiniLLM: On-Policy Distillation of Large Language Models
  7. 2026OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Models
  8. 2024DistiLLM: Towards Streamlined Distillation for Large Language Models