Rethinking the Generation Order of Block Diffusion Language Models
On the map: LLM reasoning → Diffusion reasoning and test-time scaling
also in 1 more
Abstract (arXiv)
Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language models (BDLMs). We show empirically and analytically that these models are naturally more aligned with left-to-right decoding than MDMs. Based on this observation, we propose Parallel Autoregressive Decoding (PARD), a simple training-free sampling method that preserves left-to-right unmasking structure while allowing parallel token commitment. Extensive experiments show that PARD consistently outperforms existing parallel samplers in generation quality, while achieving substantial speedups over pure AR decoding with only a small quality gap.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 23 of its 46 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
23 of 46 references.
Closest ancestors · 2
Same direction, newest first
- 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?26 Feb
- 2026The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models21 Jan
Other references · 21
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2025Qwen3 Technical Reportcited by 8,195
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2025Large Language Diffusion Modelscited by 872
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2025Dream 7B: Diffusion Large Language Modelscited by 515
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
- 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
9 more
- 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
- 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
- 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
- 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
- 2025A Survey on Diffusion Language Modelscited by 75
- 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editingcited by 48
- 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMscited by 22
- 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 8
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation
- 2026Locally Coherent Parallel Decoding in Diffusion Language Models
- 2026Set Diffusion: Interpolating Token Orderings Between Autoregression and Diffusion for Fast and Flexible Decoding
- 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
- 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
- 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
- 2025On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond
- 2026Mean-Field Parallel Decoding for Discrete Diffusion Language Models