mlsanityData to 14 Oct 2026

dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching

arXiv 2506.06295 ↗First posted 17 May 2025cs.LG cs.AI cs.CLcited by 169

Not on the map. Why?

Cited by 89 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models. Recently, a new paradigm has emerged in the form of diffusion-based Large Language Models (dLLMs), which generate text by iteratively denoising masked segments. This approach has shown significant advantages and potential. However, dLLMs suffer from high inference latency. Traditional ARM acceleration techniques, such as Key-Value caching, are incompatible with dLLMs due to their bidirectional attention mechanism. To address this specific challenge, our work begins with a key observation that dLLM inference involves a static prompt and a partially dynamic response, where most tokens remain stable across adjacent denoising steps. Based on this, we propose dLLM-Cache, a training-free adaptive caching framework that combines long-interval prompt caching with partial response updates guided by feature similarity. This design enables efficient reuse of intermediate computations without compromising model performance. Extensive experiments on representative dLLMs, including LLaDA 8B and Dream 7B, show that dLLM-Cache achieves up to 9.1x FLOPs reduction on LongBench-HotpotQA while maintaining competitive output quality. Notably, our method brings dLLM inference latency close to that of ARMs under many settings. The code for this work is publicly available at: https://github.com/maomaocun/dLLM-cache.

Key facts. Cited by 169 papers in the map (data through 14 Oct 2026). Builds on 32 of its 48 references that are in the map. First posted on arXiv 17 May 2025.

Builds on

32 of 48 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Scalable Diffusion Models with Transformerscited by 6,054
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  12. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
20 more
  1. 2025Large Language Diffusion Modelscited by 912
  2. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  3. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  6. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  7. 2022Efficiently Scaling Transformer Inferencecited by 539
  8. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  9. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  10. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  11. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  12. 2023DeepCache: Accelerating Diffusion Models for Freecited by 359
  13. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  14. 2024Scaling up Masked Diffusion Models on Textcited by 199
  15. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  16. 2024Accelerating Diffusion Transformers with Token-wise Feature Cachingcited by 156
  17. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  18. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 137
  19. 2025From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeerscited by 136
  20. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 72; 2026: 97.

169 papers.

Sort:

By area:

  1. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  2. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  3. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 116
  4. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  5. 2025A Survey on Latent Reasoningcited by 81
  6. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  7. 2025A Survey on Diffusion Language Modelscited by 78
  8. 2025Diffusion Language Models Know the Answer Before Decodingcited by 70
  9. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Evictioncited by 65
  10. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  11. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  12. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 57
38 more
  1. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 46
  2. 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 45
  3. 2025DiffuSpec: Unlocking Diffusion Language Models for Speculative Decodingcited by 42
  4. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Modelscited by 41
  5. 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMscited by 40
  6. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 39
  7. 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 34
  8. 2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 34
  9. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 33
  10. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  11. 2025AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Sizecited by 31
  12. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  13. 2025LLaDA-VLA: Vision Language Diffusion Action Modelscited by 30
  14. 2025dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thoughtcited by 30
  15. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 30
  16. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 27
  17. 2025Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generationcited by 26
  18. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 25
  19. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMscited by 24
  20. 2026Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decodingcited by 24
  21. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 23
  22. 2025Set Block Decoding is a Language Model Inference Acceleratorcited by 23
  23. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Modelscited by 23
  24. 2025Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Waycited by 22
  25. 2025Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphscited by 21
  26. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagationcited by 21
  27. 2025NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matchingcited by 21
  28. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 21
  29. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 20
  30. 2025Sequential Diffusion Language Modelscited by 19
  31. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Modelscited by 19
  32. 2025ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decodingcited by 18
  33. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  34. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  35. 2025Inpainting-Guided Policy Optimization for Diffusion Large Language Modelscited by 16
  36. 2025Exploring the Potential of Diffusion Large Language Models in Code Generationcited by 16
  37. 2025Soft-Masked Diffusion Language Modelscited by 16
  38. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 16

50 most cited of 169 All 169 (JSON)