mlsanityData to 14 Oct 2026

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles

arXiv 2506.10848 ↗First posted 12 Jun 2025cs.CL cs.AI cs.LGcited by 45

Not on the map. Why?

Cited by 25 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Diffusion-based language models (dLLMs) have emerged as a promising alternative to traditional autoregressive LLMs by enabling parallel token generation and significantly reducing inference latency. However, existing sampling strategies for dLLMs, such as confidence-based or semi-autoregressive decoding, often suffer from static behavior, leading to suboptimal efficiency and limited flexibility. In this paper, we propose SlowFast Sampling, a novel dynamic sampling strategy that adaptively alternates between exploratory and accelerated decoding stages. Our method is guided by three golden principles: certainty principle, convergence principle, and positional principle, which govern when and where tokens can be confidently and efficiently decoded. We further integrate our strategy with dLLM-Cache to reduce redundant computation. Extensive experiments across benchmarks and models show that SlowFast Sampling achieves up to 15.63× speedup on LLaDA with minimal accuracy drop, and up to 34.22× when combined with caching. Notably, our approach outperforms strong autoregressive baselines like LLaMA3 8B in throughput, demonstrating that well-designed sampling can unlock the full potential of dLLMs for fast and high-quality generation.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Builds on 19 of its 23 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

19 of 23 references.

References · 19

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  9. 2023Large Language Modelscited by 780
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  11. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
7 more
  1. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  3. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  4. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 208
  5. 2024Scaling up Masked Diffusion Models on Textcited by 199
  6. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  7. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 143

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 32.

45 papers.

Sort:

By area:

  1. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  2. 2025Diffusion Language Models Know the Answer Before Decodingcited by 70
  3. 2025DPad: Efficient Diffusion Language Models with Suffix Dropoutcited by 39
  4. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  5. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 27
  6. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 18
  7. 2025Soft-Masked Diffusion Language Modelscited by 16
  8. 2025Thinking Inside the Mask: In-Place Prompting in Diffusion LLMscited by 13
  9. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Creditcited by 13
  10. 2025Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedulescited by 11
  11. 2025Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusioncited by 10
  12. 2025dUltra: Ultra-Fast Diffusion Language Models via Reinforcement Learningcited by 10
33 more
  1. 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Modelscited by 5
  2. 2026Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inferencecited by 4
  3. 2026Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Modelscited by 4
  4. 2026Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Modelscited by 4
  5. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 3
  6. 2026Just on Time: Token-Level Early Stopping for Diffusion Language Modelscited by 2
  7. 2026Drifting Objectives for Refining Discrete Diffusion Language Modelscited by 2
  8. 2026FlexDraft: Flexible Speculative Decoding via Attention Tuning and Bonus-Guided Calibrationcited by 2
  9. 2026Sangam: Efficiently Serving Diffusion LLMs with the AR Stackcited by 2
  10. 2025C2DLM: Causal Concept-Guided Diffusion Large Language Modelscited by 1
  11. 2026STDec: Spatio-Temporal Stability Guided Decoding for dLLMscited by 1
  12. 2026Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolationcited by 1
  13. 2026Supportive Token Revealing for Fast Diffusion Language Model Decodingcited by 1
  14. 2026VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Terminationcited by 1
  15. 2026Understanding Parallel Samplers in Masked Diffusion via Random Walks on Graphscited by 1
  16. 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMscited by 1
  17. 2026Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Modelscited by 1
  18. 2025STaRR: Spatial-Temporal Token-Dynamics-Aware Responsive Remasking for Diffusion Language Modelscited by 0
  19. 2026Beyond Scattered Acceptance: Fast and Coherent Inference for DLMs via Longest Stable Prefixescited by 0
  20. 2026Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrastcited by 0
  21. 2026HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrievalcited by 0
  22. 2026Understanding Evaluation Illusion in Diffusion Large Language Modelscited by 0
  23. 2026Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Modelscited by 0
  24. 2026CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inferencecited by 0
  25. 2026Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMscited by 0
  26. 2026Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Modelscited by 0
  27. 2026Survival-Guided Length Control for Efficient Diffusion Language Modelscited by 0
  28. 2026Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Modelscited by 0
  29. 2026Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMscited by 0
  30. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Samplingcited by 0
  31. 2026Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Modelscited by 0
  32. 2026Batch-Paced: Fixed-Budget Batch Decoding in Discrete Diffusion Language Modelscited by 0
  33. 2026A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMscited by 0