mlsanityData to 7 Oct 2026

Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

arXiv 2609.26796 ↗First posted 22 Sep 2026cs.CLcited by 0

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to autoregressive LLMs by enabling non-autoregressive text generation. However, their practical deployment remains limited by inefficient inference, largely due to the absence of effective Key-Value (KV) caching and scalable parallel decoding mechanisms. Existing acceleration methods typically study KV caching and parallel decoding in isolation, overlooking the I/O bottlenecks that arise when cache reuse and parallel token verification are jointly applied. In this work, we introduce Flash-dLLM, a training-free inference acceleration framework for fast and memory-efficient dLLMs. Flash-dLLM first identifies GPU memory I/O as a dominant bottleneck in KV-cache-enabled dLLM inference and addresses it with an I/O-aware fused KV-cache kernel that reduces redundant memory movement. Building on this optimized cache mechanism, Flash-dLLM further proposes an efficient KV-cache-driven draft-and-verify decoding strategy, where the dLLM itself serves as both drafter and verifier without requiring an auxiliary model. This unified design enables faster decoding while preserving generation quality and improving scalability to longer sequences and larger batch size. Extensive experiments on mathematical reasoning and code-generation benchmarks demonstrate that Flash-dLLM consistently outperforms existing state-of-the-art dLLM acceleration methods in both inference speed and memory efficiency. In particular, it achieves 5.1× and 11.0× speedups over prior strongest baseline Elastic-Cache on GSM8K and HumanEval, respectively.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 52 of its 56 references that are in the map. First posted on arXiv 22 Sep 2026.

Builds on

52 of 56 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM27 May
  2. 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models2 Apr
  3. 2025Attention Is All You Need for KV Cache in Diffusion LLMs16 Oct
  4. 2025Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference10 Oct
  5. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagation8 Oct
  6. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models30 Sep
  7. 2025Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs22 Sep
  8. 2025Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction4 Aug
  9. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding31 May
  10. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May
  11. 2025dKV-Cache: The Cache for Diffusion Language Models21 May

Other references · 41

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
29 more
  1. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  2. 2025OpenAI GPT-5 System Cardcited by 910
  3. 2025Large Language Diffusion Modelscited by 872
  4. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  6. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  7. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  9. 2022Efficiently Scaling Transformer Inferencecited by 529
  10. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  11. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  12. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  13. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  14. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  15. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  16. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  17. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  18. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 204
  19. 2024Scaling up Masked Diffusion Models on Textcited by 195
  20. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  21. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  22. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  23. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  24. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  25. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusioncited by 77
  26. 2025A Survey on Diffusion Language Modelscited by 75
  27. 2025Diffusion Language Models Know the Answer Before Decodingcited by 68
  28. 2026FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scalingcited by 58
  29. 2024A Theoretical Perspective for Speculative Decoding Algorithmcited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
  2. 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
  3. 2025Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Models
  4. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
  6. 2026GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference
  7. 2026Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
  8. 2026Sangam: Efficiently Serving Diffusion LLMs with the AR Stack