mlsanityData to 7 Oct 2026

dParallel: Learnable Parallel Decoding for dLLMs

arXiv 2509.26488 ↗First posted 30 Sep 2025cs.CLcited by 59

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion large language models (dLLMs) have recently drawn considerable attention within the research community as a promising alternative to autoregressive generation, offering parallel token prediction and lower inference latency. Yet, their parallel decoding potential remains largely underexplored, as existing open-source models still require nearly token-length decoding steps to ensure performance. To address this, we introduce dParallel, a simple and effective method that unlocks the inherent parallelism of dLLMs for fast sampling. We identify that the key bottleneck to parallel decoding arises from the sequential certainty convergence for masked tokens. Building on this insight, we introduce the core of our approach: certainty-forcing distillation, a novel training strategy that distills the model to follow its original sampling trajectories while enforcing it to achieve high certainty on masked tokens more rapidly and in parallel. Extensive experiments across various benchmarks demonstrate that our method can dramatically reduce the number of decoding steps while maintaining performance. When applied to the LLaDA-8B-Instruct model, dParallel reduces decoding steps from 256 to 30 on GSM8K, achieving an 8.5x speedup without performance degradation. On the MBPP benchmark, it cuts decoding steps from 256 to 24, resulting in a 10.5x speedup while maintaining accuracy. Our code is available at https://github.com/czg1225/dParallel

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 50 of its 53 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

50 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  2. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding31 May
  3. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
  4. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  5. 2025dKV-Cache: The Cache for Diffusion Language Models21 May

Other references · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2023Let's Verify Step by Stepcited by 3,598
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
33 more
  1. 2023Qwen Technical Reportcited by 3,273
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  3. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  4. 2025Large Language Diffusion Modelscited by 872
  5. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  6. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  7. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  9. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  10. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  11. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  12. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  13. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  14. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  15. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  16. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  17. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  18. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
  19. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  20. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  21. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  22. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  23. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
  24. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  25. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 80
  26. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusioncited by 77
  27. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Timecited by 73
  28. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  29. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  30. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 38
  31. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  32. 2025DLLMQuant: Quantizing Diffusion-based Large Language Modelscited by 10
  33. 2025Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 50.

59 papers.

Sort:

By area:

  1. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  2. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 34
  3. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  4. 2025Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Waycited by 22
  5. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 20
  6. 2025OBS-Diff: Accurate Pruning For Diffusion Models in One-Shotcited by 19
  7. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  8. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  9. 2026DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMscited by 13
  10. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMscited by 12
  11. 2025Fast and Accurate Causal Parallel Decoding using Jacobi Forcingcited by 11
  12. 2025dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learningcited by 10
38 more
  1. 2026DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmaskingcited by 8
  2. 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Modelscited by 8
  3. 2026DSB: Dynamic Sliding Block Scheduling for Diffusion LLMscited by 7
  4. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  5. 2026Stop the Flip-Flop: Context-Preserving Verification for Fast Revocable Diffusion Decodingcited by 6
  6. 2026MultiAnimate: Pose-Guided Image Animation Made Extensiblecited by 6
  7. 2026LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learningcited by 6
  8. 2026R2-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reductioncited by 6
  9. 2025On the Role of Discreteness in Diffusion LLMscited by 4
  10. 2026Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inferencecited by 4
  11. 2026FOCUS: DLLMs Know How to Tame Their Compute Boundcited by 3
  12. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Modelscited by 3
  13. 2026Few-Step Diffusion Language Models via Trajectory Self-Distillationcited by 3
  14. 2026Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Modelscited by 3
  15. 2026ECHO: Efficient Chest X-ray Report Generation with One-step Block Diffusioncited by 3
  16. 2026Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusioncited by 3
  17. 2026OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Modelscited by 3
  18. 2026Plan, Verify and Fill: A Structured Parallel Decoding Approach for Diffusion Language Modelscited by 2
  19. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 2
  20. 2026STRIDE: When to Speak Meets Sequence Denoising for Streaming Video Understandingcited by 2
  21. 2026TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLMcited by 2
  22. 2026Multi-Block Diffusion Language Modelscited by 2
  23. 2026DIP: Dynamic In-Context Planner For Diffusion Language Modelscited by 1
  24. 2026MedCoG: Maximizing LLM Inference Density in Medical Reasoning via Meta-Cognitive Regulationcited by 1
  25. 2026Generalized Discrete Diffusion with Self-Correctioncited by 1
  26. 2026TRIMS: Trajectory-Ranked Instruction Masked Supervision for Diffusion Language Modelscited by 1
  27. 2026DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inferencecited by 1
  28. 2026Distillation Traps and Guards: A Calibration Knob for LLM Distillabilitycited by 1
  29. 2026Layer Collapse in Diffusion Language Modelscited by 1
  30. 2026DARE: Diffusion Language Model Activation Reuse for Efficient Inferencecited by 1
  31. 2026The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Modelscited by 1
  32. 2026Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolationcited by 1
  33. 2026Efficient On-Device Diffusion LLM Inference with Mobile NPUcited by 1
  34. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Modelscited by 1
  35. 2026LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detectioncited by 0
  36. 2026PulseCol: Periodically Refreshed Column-Sparse Attention for Accelerating Diffusion Language Modelscited by 0
  37. 2026Understanding Evaluation Illusion in Diffusion Large Language Modelscited by 0
  38. 2026DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decodingcited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models
  2. 2026CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing
  3. 2026Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding
  4. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
  5. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
  6. 2026Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
  7. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
  8. 2026dVoting: Fast Voting for dLLMs