mlsanityData to 7 Oct 2026

Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing

arXiv 2508.09192 ↗First posted 8 Aug 2025cs.LG cs.AIcited by 111

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs for text generation, with the potential to decode multiple tokens in a single iteration. However, none of the existing open-source dLLMs have achieved superior inference speed over AR LLMs of similar size. This paper breaks this barrier based on a simple and effective strategy named discrete diffusion forcing (D2F). D2F equips dLLMs with two key capabilities: (1) block-wise autoregressive generation to enable KV cache utilization; (2) prediction of following tokens without requiring completion of prior blocks for inter-block parallel decoding. In this way, the vanilla dLLMs are refurbished into an AR-diffusion hybrid paradigm for efficient inference. D2F can be implemented with an asymmetric distillation process based on pre-trained dLLMs. We further propose a pipelined parallel decoding algorithm, which enables a trade-off between efficiency and efficacy. Empirically, D2F dLLMs achieve more than 2.5× inference speed than LLaMA3 and Qwen2.5 on GSM8K. Compared to vanilla dLLMs like LLaDA and Dream, the acceleration can be more than 50× while maintaining comparable output quality. The code is available at https://github.com/zhijie-group/Discrete-Diffusion-Forcing.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 38 of its 41 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

38 of 41 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  2. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  3. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  4. 2025Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithms10 Mar
  5. 2025Large Language Diffusion Models14 Feb
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 32

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2024Qwen2.5 Technical Reportcited by 4,713
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  12. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
20 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,509
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  4. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  5. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  6. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  7. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  8. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  10. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  11. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
  12. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  13. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 262
  14. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  15. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  16. 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 159
  17. 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 157
  18. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  19. 2025Long-Context State-Space Video World Modelscited by 59
  20. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusioncited by 53

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 81.

111 papers.

Sort:

By area:

  1. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  2. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  3. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  4. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 59
  5. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 38
  6. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
  7. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  8. 2025Scaling Behavior of Discrete Diffusion Language Modelscited by 32
  9. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 30
  10. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 30
  11. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  12. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
38 more
  1. 2025Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Waycited by 22
  2. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMscited by 22
  3. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  4. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 20
  5. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 20
  6. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 19
  7. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  8. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  9. 2025Exploring the Potential of Diffusion Large Language Models in Code Generationcited by 16
  10. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decodingcited by 16
  11. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  12. 2026DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learningcited by 13
  13. 2025SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understandingcited by 12
  14. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcingcited by 11
  15. 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?cited by 11
  16. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practicescited by 10
  17. 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Modelscited by 10
  18. 2026Residual Context Diffusion Language Modelscited by 9
  19. 2026Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performancecited by 9
  20. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Accelerationcited by 8
  21. 2025Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Modelscited by 7
  22. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  23. 2025Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Modelscited by 6
  24. 2026Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Modelscited by 6
  25. 2026The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Checkcited by 6
  26. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 6
  27. 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decodingcited by 6
  28. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  29. 2026R2-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reductioncited by 6
  30. 2025Watermarking Discrete Diffusion Language Modelscited by 5
  31. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculationcited by 5
  32. 2025DiffuMamba: High-Throughput Diffusion LMs with Mamba Backbonecited by 4
  33. 2025Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Modelscited by 3
  34. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Modelscited by 3
  35. 2026Few-Step Diffusion Language Models via Trajectory Self-Distillationcited by 3
  36. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 3
  37. 2026Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Modelscited by 3
  38. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 3

50 most cited of 111 All 111 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Discrete Diffusion in Large Language and Multimodal Models: A Survey
  2. 2026Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
  3. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
  4. 2026Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
  5. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  6. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusion
  7. 2025A Survey on Diffusion Language Models
  8. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding