mlsanityData to 7 Oct 2026

Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models

arXiv 2604.08964 ↗First posted 10 Apr 2026cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Diffusion Large Language Models (dLLMs) have recently become a promising alternative to autoregressive large language models (ARMs). Semi-autoregressive (Semi-AR) decoding is widely employed in base dLLMs and advanced decoding strategies due to its superior performance. However, our observations reveal that Semi-AR decoding suffers from inherent block constraints, which cause the decoding of many cross-block stable tokens to be unnecessarily delayed. To address this challenge, we systematically investigate the identification of stable tokens and present three key findings: (1) naive lookahead decoding is unreliable, (2) token stability closely correlates with convergence trend, and (3) historical information is isolated. Building on these insights, we propose Anchor-based History-stable Decoding (AHD), a training-free, plug-and-play dynamic decoding strategy. Specifically, AHD monitors the stability trend of tokens in real time through dynamic anchors. Once a token reaches stability, it initiates early cross-block decoding to enhance efficiency and performance. Extensive experiments across language, vision-language, and audio-language domains demonstrate that AHD simultaneously improves both performance and inference efficiency. Notably, AHD effectively reverses the performance degradation typically observed in existing advanced decoding acceleration strategies. For instance, on the BBH benchmark, our approach reduces decoding steps by 80% while improving performance by 3.67%.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

35 of 48 references.

References · 35

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  11. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
23 more
  1. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  2. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  3. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  4. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  5. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  7. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
  8. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  9. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 111
  10. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  11. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  12. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 64
  13. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  14. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 45
  15. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  16. 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 43
  17. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 38
  18. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 30
  19. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noisecited by 21
  20. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagationcited by 21
  21. 2025Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inferencecited by 17
  22. 2025Saber: An Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Modelcited by 13
  23. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrastcited by 0
  2. 2026Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMscited by 0
  3. 2026Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMscited by 0