mlsanityData to 7 Oct 2026

Accelerated Diffusion Models via Speculative Sampling

arXiv 2501.05370 ↗First posted 9 Jan 2025cs.LG stat.MLcited by 30

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Speculative sampling is a popular technique for accelerating inference in Large Language Models by generating candidate tokens using a fast draft model and accepting or rejecting them based on the target model's distribution. While speculative sampling was previously limited to discrete sequences, we extend it to diffusion models, which generate samples via continuous, vector-valued Markov chains. In this context, the target model is a high-quality but computationally expensive diffusion model. We propose various drafting strategies, including a simple and effective approach that does not require training a draft model and is applicable out of the box to any diffusion model. Our experiments demonstrate significant generation speedup on various diffusion models, halving the number of function evaluations, while generating exact samples from the target model.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 61 of its 77 references that are in the map. First posted on arXiv 9 Jan 2025.

Builds on

61 of 77 references.

References · 61

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  11. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  12. 2023Consistency Modelscited by 1,778
38 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  3. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  4. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  5. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  6. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  7. 2022On Distillation of Guided Diffusion Modelscited by 721
  8. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  9. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  10. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  11. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  12. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 448
  13. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  14. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  15. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  16. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 344
  17. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  18. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
  19. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 266
  20. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
  21. 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 195
  22. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
  23. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  24. 2023Accelerating LLM Inference with Staged Speculative Decodingcited by 151
  25. 2018Survey of multifidelity methods in uncertainty propagation, inference, and optimizationcited by 141
  26. 2023SpecTr: Fast Speculative Decoding via Optimal Transportcited by 129
  27. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  28. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillationcited by 115
  29. 2023Parallel Sampling of Diffusion Modelscited by 112
  30. 2023Noise-Free Score Distillationcited by 99
  31. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Modelscited by 98
  32. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Modelscited by 91
  33. 2019The Randomized Midpoint Method for Log-Concave Samplingcited by 77
  34. 2024Multistep Distillation of Diffusion Models via Moment Matchingcited by 76
  35. 2022Wavelet Score-Based Generative Modelingcited by 61
  36. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusioncited by 57
  37. 2021Instantaneous Grammatical Error Correction with Shallow Aggressive Decodingcited by 53
  38. 2024Target Score Matchingcited by 46

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 15.

30 papers.

Sort:

By area:

  1. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  2. 2025How to build a consistency model: Learning flow maps via self-distillationcited by 96
  3. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  4. 2025Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphscited by 21
  5. 2025Playing with Transformer at 30+ FPS via Next-Frame Diffusioncited by 18
  6. 2025Lipschitz-Guided Design of Interpolation Schedules in Generative Modelscited by 15
  7. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 14
  8. 2025Diffusion Models are Secretly Exchangeable: Parallelizing DDPMs via Autospeculationcited by 11
  9. 2025Generalised Flow Maps for Few-Step Generative Modelling on Riemannian Manifoldscited by 10
  10. 2026Realtime-VLA FLASH: Speculative Inference Framework for Diffusion-based VLAscited by 8
  11. 2025TS-DP: Reinforcement Speculative Decoding For Temporal Adaptive Diffusion Policy Accelerationcited by 6
  12. 2025List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compressioncited by 3
18 more
  1. 2025Dimension-free error estimate for diffusion model and optimal schedulingcited by 3
  2. 2025A Comprehensive Review of Diffusion Models in Smart Agriculture: Progress, Applications, and Challengescited by 2
  3. 2026Accelerating Speculative Diffusions via Block Verificationcited by 2
  4. 2026CachedSearch: Training-Free Cached Exploration for Test-Time Search in Video Diffusioncited by 2
  5. 2026Accelerating Diffusion Sampling via Speculative Draft Treescited by 2
  6. 2025Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorationscited by 1
  7. 2025FREE: Uncertainty-Aware Autoregression for Parallel Diffusion Transformerscited by 1
  8. 2026Speculative Sampling For Faster Molecular Dynamicscited by 1
  9. 2026Improved Distributional Diffusion Modelscited by 1
  10. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  11. 2026Draft-and-Target Sampling for Video Generation Policycited by 0
  12. 2026BadDLM: Backdooring Diffusion Language Models with Diverse Targetscited by 0
  13. 2026PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decodingcited by 0
  14. 2026Trajectory-Level Speculative Decoding for Diffusion Language Modelscited by 0
  15. 2026Membership Inference in Fine-tuned Diffusion Language Models via Token-level Memorization Asymmetrycited by 0
  16. 2026Watermarkable Multi-Draft Speculative Sampling via Poisson Processescited by 0
  17. 2026ReAL: Accelerating Flow Matching through Segment Advancement with Shared Lookaheadcited by 0
  18. 2026Rank-Aware Speculative Sampling for Diffusion Draft Treescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
  2. 2025Diffusion Language Models are Provably Optimal Parallel Samplers
  3. 2025Optimal Inference Schedules for Masked Diffusion Models
  4. 2025Dream 7B: Diffusion Large Language Models
  5. 2023Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time
  6. 2025Diffusion Language Model Inference with Monte Carlo Tree Search
  7. 2026SimSD: Simple Speculative Decoding in Diffusion Language Models
  8. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion