BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference
Abstract (arXiv)
Efficient serving of diffusion large language models (dLLMs) is hindered by convergence heterogeneity: when batching multiple requests, different sequences converge at different rates, causing faster requests to stall behind slower stragglers and introducing compute bubbles and tail latency. We present BlockServe, a continuous batching framework that integrates block-grained scheduling -- immediately evicting completed requests at block boundaries -- with mixed-state execution that extends dual cache and parallel decoding to heterogeneous batches via gather-scatter indexing. Furthermore, a compute-aware admission controller expands effective batch capacity through token-budgeted refill. On Dream and LLaDA across five benchmarks, BlockServe achieves 1.9--10.6× throughput over Fast-dLLM with comparable generation quality, establishing block-grained scheduling as a foundation for high-throughput offline dLLM inference.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 20 of its 21 references that are in the map. First posted on arXiv 9 Jul 2026.
Builds on
20 of 21 references.
Closest ancestors · 1
Same direction, newest first
- 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
Other references · 19
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
- 2025Large Language Diffusion Modelscited by 872
- 2024Simple and Effective Masked Diffusion Language Modelscited by 800
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
- 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
7 more
- 2025Dream 7B: Diffusion Large Language Modelscited by 515
- 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 507
- 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 187
- 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
- 2024DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inferencecited by 106
- 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
- 2025Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM Servingcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DSB: Dynamic Sliding Block Scheduling for Diffusion LLMs
- 2024Multi-Bin Batching for Increasing LLM Inference Throughput
- 2026BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
- 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
- 2026SynerDiff: Synergetic Continuous Batching for Fast and Parallel Diffusion Model Inference
- 2025BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
- 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
- 2025Fast-dLLM v2: Efficient Block-Diffusion LLM