mlsanityData to 7 Oct 2026

d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models

arXiv 2609.35362 ↗First posted 28 Sep 2026cs.LG cs.AIcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Large language models (LLMs) typically generate text autoregressively (AR), predicting one token at a time. Block diffusion language models (dLLMs) instead generate blocks sequentially while denoising multiple tokens in parallel within each block, offering a promising way to accelerate generation. Rather than training such models from scratch, recent work adapts strong pretrained AR models into block dLLMs through distillation. On-policy distillation (OPD) has been widely used for LLM training because it supervises the student on states generated by its current policy, rather than only on fixed offline trajectories. By training on the states the student actually visits, it reduces the mismatch between training and generation and can provide more relevant supervision as the student evolves. Recent work has extended this idea to AR-to-block-diffusion conversion. However, this setting introduces a fundamental mismatch in supervision: the block-diffusion student and the causal AR teacher condition on different information at the same training state. The student predicts from the entire partially denoised block, including visible future context, whereas the standard AR teacher target is defined only from the causal prefix. As a result, the teacher distribution used for distillation is not fully aligned with the information available to the student. We therefore introduce d-OPD, a future-aware on-policy distillation method that corrects the AR teacher distribution to better align with the student-visible state by incorporating visible future information within each block, providing supervision that better matches the information used by the student. Across Qwen3 models from 0.6B to 8B, d-OPD improves the six-benchmark average by up to 4.0 points over OPDLM and reduces training time by 1.35-1.58×. The code is available at https://github.com/mit-han-lab/d-OPD.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 69 of its 77 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

69 of 77 references.

Closest ancestors · 31

Same direction, newest first

  1. 2026Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale9 Sep
  2. 2026Rethinking the Generation Order of Block Diffusion Language Models27 Jul
  3. 2026FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models20 Jul
  4. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques14 Jul
  5. 2026Sangam: Efficiently Serving Diffusion LLMs with the AR Stack5 Jul
  6. 2026Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models29 Jun
  7. 2026Multi-Block Diffusion Language Models28 Jun
  8. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation4 Jun
  9. 2026Cluster-Level Attention-Guided Parallel Decoding for Masked Diffusion Language Models28 May
  10. 2026From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons11 Apr
  11. 2026DMax: Aggressive Parallel Decoding for dLLMs9 Apr
  12. 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?26 Feb
19 more
  1. 2026Diffusion In Diffusion: Reclaiming Global Coherence in Semi-Autoregressive Diffusion20 Jan
  2. 2026T^star: Progressive Block Scaling for Masked Diffusion Language Models Through Trajectory Aware Reinforcement Learning16 Jan
  3. 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Models23 Dec
  4. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed16 Dec
  5. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding15 Dec
  6. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs7 Dec
  7. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models26 Nov
  8. 2025Planned Diffusion20 Oct
  9. 2025Soft-Masked Diffusion Language Models20 Oct
  10. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  11. 2025Sequential Diffusion Language Models28 Sep
  12. 2025Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning18 Sep
  13. 2025Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding27 Aug
  14. 2025Dream 7B: Diffusion Large Language Models21 Aug
  15. 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey16 Jun
  16. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  17. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  18. 2025Large Language Diffusion Models14 Feb
  19. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct

Other references · 38

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  7. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  8. 2020Aligning AI With Shared Human Valuescited by 706
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  10. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  11. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  12. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 262
26 more
  1. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  2. 2026A Survey of On-Policy Distillation for Large Language Modelscited by 148
  3. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  4. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  5. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  6. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 55
  7. 2025A Comprehensive Survey on Knowledge Distillationcited by 49
  8. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  9. 2026OPSDL: On-Policy Self-Distillation for Long-Context Language Modelscited by 29
  10. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 26
  11. 2025RLP: Reinforcement as a Pretraining Objectivecited by 23
  12. 2026On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agentscited by 21
  13. 2026Rethinking On-Policy Distillation of Large Language Models II: One Training Examplecited by 15
  14. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Modelscited by 13
  15. 2025SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understandingcited by 12
  16. 2026On-Policy Self-Distillation without Any Supervisioncited by 11
  17. 2025DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Modelscited by 10
  18. 2026Self-Supervised On-Policy Distillation for Reasoning Language Modelscited by 10
  19. 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 8
  20. 2025Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Predictioncited by 7
  21. 2025Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Predictioncited by 5
  22. 2025Reasoning Bias of Next Token Prediction Trainingcited by 3
  23. 2025Sequence-level Large Language Model Training with Contrastive Preference Optimizationcited by 3
  24. 2026BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillationcited by 2
  25. 2026GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inferencecited by 2
  26. 2026K-Forcing: Joint Next-K-Token Decoding via Push-Forward Language Modelingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning from the Self-future: On-policy Self-distillation for dLLMs
  2. 2026dOPSD: On-Policy Self-Distillation for Diffusion Language Models
  3. 2026Towards Unbiased On-Policy Distillation for Block Diffusion Language Models
  4. 2026GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales
  5. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models
  6. 2026Trace-Based On-Policy Distillation for Masked Diffusion Language Models
  7. 2026OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Models
  8. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe