Mean-Field Parallel Decoding for Discrete Diffusion Language Models
On the map: Diffusion models → Discrete and text diffusion
also in 1 more
Abstract (arXiv)
Discrete diffusion language models enable parallel token generation, offering a pathway to low-latency decoding. However, selecting tokens independently by marginal confidence limits effective parallelism: tokens that appear reliable in isolation can form incompatible configurations when several positions are updated at once. We introduce a training-free decoding framework that coordinates these parallel updates. At each forward pass, the method assigns a commit score to each masked position and refines these scores using pairwise interactions derived from the model's predictive distributions. A variational relaxation yields a simple fixed-point update that suppresses conflicting simultaneous commitments within a single forward pass. This mechanism allows the decoder to commit more tokens in parallel while maintaining competitive generation quality. The method is lightweight, requires no auxiliary model or retraining, and drops into existing diffusion decoding pipelines without modification. Experiments on reasoning and code-generation benchmarks show consistent improvements in the quality-latency trade-off.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 26 of its 32 references that are in the map. First posted on arXiv 14 Jun 2026.
Builds on
26 of 32 references.
Closest ancestors · 7
Same direction, newest first
- 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models2 Apr
- 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editing9 Feb
- 2026DFlash: Block Diffusion for Flash Speculative Decoding5 Feb
- 2026Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective30 Jan
- 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
- 2025Dream 7B: Diffusion Large Language Models21 Aug
- 2025Large Language Diffusion Models14 Feb
Other references · 19
- 2023GPT-4 Technical Reportcited by 18,396
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Qwen Technical Reportcited by 3,344
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
- 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
- 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
7 more
- 2017Non-Autoregressive Neural Machine Translationcited by 463
- 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
- 2026Accelerating Speculative Decoding with Block Diffusion Draft Treescited by 29
- 2025Accelerating Diffusion LLM Inference via Local Determinism Propagationcited by 21
- 2026DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMscited by 14
- 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMscited by 13
- 2025Speculative Verification: Exploiting Information Gain to Refine Speculative Decodingcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reliable Parallel Decoding in Masked Diffusion Language Models
- 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
- 2026Rethinking Soft Tokens for Parallel Decoding in Diffusion Language Models
- 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow
- 2026Locally Coherent Parallel Decoding in Diffusion Language Models
- 2026Edit-Based Refinement for Parallel Masked Diffusion Language Models
- 2026Supportive Token Revealing for Fast Diffusion Language Model Decoding
- 2026Stability-Weighted Decoding for Diffusion Language Models