mlsanityData to 7 Oct 2026

Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models

arXiv 2510.00294 ↗First posted 30 Sep 2025cs.LG cs.AIcited by 19

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion Large Language Models (DLLMs) have emerged as a new paradigm of language modeling beyond autoregressive next-token prediction. Taking advantage of their inherent modeling foundations, DLLMs have the great potential of efficient inference with parallel decoding algorithms, which enable multi-token prediction. However, the high generation quality often requires the number of decoding steps equal to the sequence length, which performs a one-token-per-step decoding, and existing parallel decoding algorithms, which yield suboptimal decoding paths, bring inference speedup at the cost of non-negligible performance degradation. To overcome this challenge, we introduce Free Draft-and-Verification (FreeDave), a novel fast decoding algorithm tailored for DLLMs that achieves lossless parallel decoding without any model modification or extra modules. Specifically, we propose an algorithm of parallel-decoded candidate generation and verification, which is theoretically guaranteed to use the fewest model forward calls to reproduce the same sequence generated by one-token-per-step decoding. By extensive evaluations on math reasoning and code generation benchmarks across different DLLMs, FreeDave is proven to accelerate the inference up to 2.83× without performance degradation.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 38 of its 43 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

38 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding31 May
  2. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
  3. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
23 more
  1. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  3. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  4. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  5. 2025Large Language Diffusion Modelscited by 872
  6. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  7. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  8. 2024A Survey on Large Language Models for Code Generationcited by 646
  9. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  11. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  12. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  13. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  14. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  15. 2024Scaling up Masked Diffusion Models on Textcited by 195
  16. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
  17. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  18. 2023Accelerating LLM Inference with Staged Speculative Decodingcited by 151
  19. 2024CodeGemma: Open Code Models Based on Gemmacited by 145
  20. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 80
  21. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusioncited by 57
  22. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  23. 2025Training Optimal Large Diffusion Language Modelscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 15.

19 papers.

Sort:

By area:

  1. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
  2. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  3. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  4. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  5. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  6. 2025Corrective Diffusion Language Modelscited by 11
  7. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Accelerationcited by 8
  8. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 6
  9. 2026Introspective Diffusion Language Modelscited by 6
  10. 2026Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inferencecited by 4
  11. 2026Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Modelscited by 3
  12. 2026Plan, Verify and Fill: A Structured Parallel Decoding Approach for Diffusion Language Modelscited by 2
7 more
  1. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 2
  2. 2026TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLMcited by 2
  3. 2026Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Modelscited by 1
  4. 2026Understanding Evaluation Illusion in Diffusion Large Language Modelscited by 0
  5. 2026Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMscited by 0
  6. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Modelscited by 0
  7. 2026FastGuide: Accelerating Reward Guidance for Diffusion Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding
  2. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
  3. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  4. 2026Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
  5. 2026Fast-dLLM++: Fr'echet Profile Decoding for Faster Diffusion LLM Inference
  6. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
  7. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagation
  8. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models