mlsanityData to 7 Oct 2026

Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM

arXiv 2606.26120 ↗First posted 27 May 2026cs.CL cs.LGcited by 4

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms. However, their computational complexity scales on the order of L cubed with the sequence length L. This poses significant challenges for long-sequence and real-time applications, primarily due to the lack of compatibility with key-value caching and the non-autoregressive nature of denoising steps. Existing acceleration methods rely on static caching or parallel decoding strategies, which fail to account for the dynamic behavior of token properties across layers and decoding steps. We propose Dynamic-dLLM, a training-free framework that enhances dLLM inference efficiency through two components: Dynamic Cache Updating (DCU), which adaptively allocates cache-update budgets based on layer-wise token dynamics, and Adaptive Parallel Decoding (APD), which dynamically calibrates decoding thresholds to balance generation quality and efficiency. Extensive experiments on models like LLaDA-8B-Instruct, LLaDA-1.5, and Dream-v0-7B-Instruct across benchmarks such as MMLU, GSM8K, and HumanEval demonstrate that Dynamic-dLLM significantly improves inference speed. It attains an average speedup exceeding 3 times while maintaining performance. Dynamic-dLLM outperforms state-of-the-art acceleration methods and provides a plug-and-play solution for efficient dLLM deployment without compromising performance. The code is available at https://github.com/TianyiWu233/DYNAMIC-DLLM.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 43 of its 46 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

43 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction4 Aug
  2. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
  3. 2025dKV-Cache: The Cache for Diffusion Language Models21 May

Other references · 40

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  11. 2025Large Language Diffusion Modelscited by 872
  12. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
28 more
  1. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  2. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  3. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298
  4. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  5. 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
  6. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 265
  7. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 245
  8. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  9. 2021Semi-supervised Semantic Segmentation with Directional Context-aware Consistencycited by 112
  10. 2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
  11. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  12. 2021Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentationcited by 87
  13. 2023Hierarchical Dense Correlation Distillation for Few-Shot Segmentationcited by 84
  14. 2024Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentationcited by 61
  15. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 46
  16. 2022Generalized Parametric Contrastive Learningcited by 45
  17. 2024OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentationcited by 43
  18. 2025Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representationscited by 42
  19. 2025DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perceptioncited by 33
  20. 2021PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Maskcited by 31
  21. 2024Unified Language-driven Zero-shot Domain Adaptationcited by 30
  22. 2024Scalable Language Model with Generalized Continual Learningcited by 28
  23. 2025Mitigating Object Hallucinations via Sentence-Level Early Interventioncited by 28
  24. 2024GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understandingcited by 26
  25. 2023Learning Context-aware Classifier for Semantic Segmentationcited by 23
  26. 2023Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancementcited by 19
  27. 2025Edit360: 2D Image Edits to 3D Assets from Any Anglecited by 13
  28. 2025Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perceptioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Terminationcited by 1
  2. 2026Understanding Evaluation Illusion in Diffusion Large Language Modelscited by 0
  3. 2026Affix Cache for Diffusion Large Language Modelscited by 0
  4. 2026Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
  2. 2025Fast-dLLM v2: Efficient Block-Diffusion LLM
  3. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques
  4. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
  6. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding
  7. 2026ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
  8. 2025Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference