BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
On the map: Diffusion models → Discrete and text diffusion
also in 1 more
Abstract (arXiv)
Diffusion language models (dLLMs) generate text by iteratively denoising multiple token positions in parallel, offering an attractive alternative to strictly autoregressive decoding. In practice, however, block-wise dLLM inference exposes a difficult granularity trade-off: small blocks preserve local conditioning but require many denoising steps, whereas large blocks expose more parallelism but can make premature commitments and accumulate cache error. Existing acceleration methods typically choose a single block size per request, leaving the complementarity among block sizes unused. We show that block size itself is a useful branching dimension. Different block sizes induce related but non-identical KV-cache trajectories: branches often share an initial prefix, bifurcate at semantically decisive positions, and later agree on syntactically lightweight tokens. Motivated by this structure, we propose BlockBatch, a training-free online inference framework that executes multiple block-size branches for the same request inside a batched forward pass. BlockBatch coordinates these branches through confidence-gated token merging, leader-based synchronization, and periodic full-sequence refreshes that re-anchor local block updates to a globally consistent KV state. Across 3 representative dLLMs and 4 datasets, BlockBatch reduces denoising NFEs by 26.6% on average and achieves a 1.33× average end-to-end speedup over Fast-dLLM while preserving accuracy. These results identify block-size diversity as a practical and previously underexplored axis for branch-parallel dLLM inference.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 34 of its 42 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
34 of 42 references.
Closest ancestors · 5
Same direction, newest first
- 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
- 2025Diffusion Beats Autoregressive in Data-Constrained Settings21 Jul
- 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
- 2024Scaling up Masked Diffusion Models on Text24 Oct
- 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
Other references · 29
- 2017Attention Is All You Needcited by 59,932
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
17 more
- 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
- 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
- 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
- 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
- 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
- 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
- 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
- 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
- 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
- 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
- 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
- 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 242
- 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
- 2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspectivecited by 23
- 2025Accelerating Diffusion LLM Inference via Local Determinism Propagationcited by 21
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inference
- 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
- 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation
- 2026AsyncLane: Decoupling Refinement from Advancement in Diffusion Language Model Decoding
- 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
- 2026PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models
- 2026Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
- 2026Rethinking the Generation Order of Block Diffusion Language Models