mlsanityData to 14 Oct 2026

One Block Ahead: Promoting Sampler State in Diffusion Language Models

arXiv 2610.19510 ↗First posted 14 Oct 2026cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Block-diffusion language models denoise a whole block of token positions in parallel so each forward pass can emit many tokens. However, blocks are still denoised in strict sequence, a cost that within-block acceleration leaves untouched. Here, we investigate decoding blocks concurrently and propose One Block Ahead (OBA), a training-free decoding algorithm that denoises the next block alongside the current one. We demonstrate that for uniform-state diffusion LMs, where a block's denoising progress is not visible in its intermediate tokens, concurrent decoding is advantageous if the next block's tokens are promoted together with the corresponding sampler state, including the self-conditioning signal and the denoising time. We also derive a break-even criterion that relates the benefit of OBA to the relative cost of a paired-block forward pass. On DiffusionGemma, a state-of-the-art diffusion LM, OBA increases the tokens per forward pass by 42.3% (from 15.64 to 22.25) and the single-request denoising throughput by 16.3%, averaged over ten benchmarks, with a small accuracy regression.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 29 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

27 of 29 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models20 Jul
  2. 2026Multi-Block Diffusion Language Models28 Jun
  3. 2026Sumi: Open Uniform Diffusion Language Model from Scratch17 Jun
  4. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editing9 Feb
  5. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  6. 2025Dream 7B: Diffusion Large Language Models21 Aug
  7. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  8. 2025The Diffusion Duality12 Jun
  9. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  10. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  11. 2025Large Language Diffusion Models14 Feb
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
1 more
  1. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 14

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  9. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  10. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  11. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  12. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
2 more
  1. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  2. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 24

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation
  2. 2026Rethinking the Generation Order of Block Diffusion Language Models
  3. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
  4. 2026Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale
  5. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
  6. 2026Denoising Blocks, Not Tokens: Efficient Compressed Continuous Diffusion with Branching Token Realization
  7. 2026BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
  8. 2026A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models