mlsanityData to 14 Oct 2026

HaloBlock: Faster and Better Decoding for Diffusion Large Language Models

arXiv 2610.17948 ↗First posted 13 Oct 2026cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Diffusion large language models (dLLMs) have attracted increasing research attention due to their bidirectional attention and arbitrary order decoding. However, there are two issues in dLLMs inference: the decoding difficulty is dominated by the tokens on the right side of the block, and the block-wise decoding inherently suffers from boundary contextual discontinuity. To tackle these limitations, we propose HaloBlock, a difficulty-aware adaptive halo mechanism for dLLM inference. We first leverage bidirectional cumulative entropy to characterize block decoding difficulty. Then, based on the bidirectional decoding difficulty, we adaptively allocate bidirectional halo regions on both sides of each block to mitigate boundary contextual discontinuity. The difficulty-aware design of HaloBlock smooths cross-block context propagation and lowers decoding difficulty for hard tokens, yielding better generation quality and faster convergence. Evaluated on LLaDA-8B-Instruct, Dream-7B-Instruct and LLaDA-1.5 across mathematical reasoning and code generation benchmarks, HaloBlock demonstrates consistent accuracy improvements and higher inference efficiency. Source codes are publicly available at https://github.com/PolyU-VCLab/HaloBlock.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 32 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

32 of 32 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2025Large Language Diffusion Modelscited by 912
  11. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  12. 2025Dream 7B: Diffusion Large Language Modelscited by 544
20 more
  1. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  3. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  4. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  5. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  6. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
  7. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 143
  8. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 129
  9. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 118
  10. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 116
  11. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  12. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  13. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 46
  14. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 39
  15. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 37
  16. 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 34
  17. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 31
  18. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 20
  19. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 7
  20. 2025EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fast-dLLM v2: Efficient Block-Diffusion LLM
  2. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
  3. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  4. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Models
  5. 2025Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
  6. 2026DepCap: Adaptive Block-Wise Parallel Decoding for Efficient Diffusion LM Inference
  7. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models
  8. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models