# Rethinking the Generation Order of Block Diffusion Language Models

arXiv: https://arxiv.org/abs/2607.24306 · HTML: https://mlsanity.com/paper/2607.24306/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning.
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 27 Jul 2026.

## Placed in

- LLM reasoning / [Diffusion reasoning and test-time scaling](https://mlsanity.com/lr/diffusion-reasoning-and-test-time-scaling/)
- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 23 of 23)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding](https://mlsanity.com/paper/2505.22618/) (2025)
- [LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models](https://mlsanity.com/paper/2505.19223/) (2025)
- [Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference](https://mlsanity.com/paper/2508.02193/) (2025)
- [LLaDA2.0: Scaling Up Diffusion Language Models to 100B](https://mlsanity.com/paper/2512.15745/) (2025)
- [Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking](https://mlsanity.com/paper/2505.24857/) (2025)
- [SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation](https://mlsanity.com/paper/2510.06303/) (2025)
- [Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding](https://mlsanity.com/paper/2505.16990/) (2025)
- [A Survey on Diffusion Language Models](https://mlsanity.com/paper/2508.10875/) (2025)
- [LLaDA2.1: Speeding Up Text Diffusion via Token Editing](https://mlsanity.com/paper/2602.08676/) (2026)
- [From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs](https://mlsanity.com/paper/2512.06776/) (2025)
- [The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models](https://mlsanity.com/paper/2601.15165/) (2026)
- [Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?](https://mlsanity.com/paper/2602.23225/) (2026)
- [Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM](https://mlsanity.com/paper/2604.06832/) (2026)

## Built on by (most cited first, 2 of 2)

- [Reducing Pretraining-Generation Mismatch in Diffusion Language Models](https://mlsanity.com/paper/2608.09424/) (2026)
- [d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models](https://mlsanity.com/paper/2609.35362/) (2026)

## Abstract (arXiv)

Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language models (BDLMs). We show empirically and analytically that these models are naturally more aligned with left-to-right decoding than MDMs. Based on this observation, we propose Parallel Autoregressive Decoding (PARD), a simple training-free sampling method that preserves left-to-right unmasking structure while allowing parallel token commitment. Extensive experiments show that PARD consistently outperforms existing parallel samplers in generation quality, while achieving substantial speedups over pure AR decoding with only a small quality gap.