mlsanityData to 14 Oct 2026

Parallel Sampling of Diffusion Models

arXiv 2305.16317 ↗First posted 25 May 2023cs.LG cs.AIcited by 112

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models are powerful generative models but suffer from slow sampling, often taking 1000 sequential denoising steps for one sample. As a result, considerable efforts have been directed toward reducing the number of denoising steps, but these methods hurt sample quality. Instead of reducing the number of denoising steps (trading quality for speed), in this paper we explore an orthogonal approach: can we run the denoising steps in parallel (trading compute for speed)? In spite of the sequential nature of the denoising steps, we show that surprisingly it is possible to parallelize sampling via Picard iterations, by guessing the solution of future denoising steps and iteratively refining until convergence. With this insight, we present ParaDiGMS, a novel method to accelerate the sampling of pretrained diffusion models by denoising multiple steps in parallel. ParaDiGMS is the first diffusion sampling method that enables trading compute for speed and is even compatible with existing fast sampling techniques such as DDIM and DPMSolver. Using ParaDiGMS, we improve sampling speed by 2-4x across a range of robotics and image generation models, giving state-of-the-art sampling speeds of 0.2s on 100-step DiffusionPolicy and 14.6s on 1000-step StableDiffusion-v2 with no measurable degradation of task reward, FID score, or CLIP score.

Key facts. Cited by 112 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 35 of its 40 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

35 of 40 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023Consistency Models2 Mar
  2. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  3. 2022On Distillation of Guided Diffusion Models6 Oct
  4. 2022Classifier-Free Diffusion Guidance26 Jul
  5. 2022gDDIM: Generalized denoising diffusion implicit models11 Jun
  6. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  7. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  8. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality11 Feb
  9. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  10. 2021Variational Diffusion Models1 Jul
  11. 2021Score-based Generative Modeling in Latent Space10 Jun
  12. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
3 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  2. 2020Denoising Diffusion Implicit Models6 Oct
  3. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  7. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  8. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  10. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  12. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 516
8 more
  1. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  2. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  3. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  4. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  5. 2020Interacting particle solutions of Fokker-Planck equations through gradient-log-density estimationcited by 88
  6. 2019The Randomized Midpoint Method for Log-Concave Samplingcited by 78
  7. 2020Accelerating Feedforward Computation via Parallel Nonlinear Equation Solvingcited by 52
  8. 2023Diffusion Model-Augmented Behavioral Cloningcited by 38

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 30; 2025: 45; 2026: 29.

112 papers.

Sort:

By area:

  1. 2023DeepCache: Accelerating Diffusion Models for Freecited by 359
  2. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 269
  3. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 217
  4. 2024Consistency Models Made Easycited by 144
  5. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 142
  6. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Modelscited by 98
  7. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  8. 2023StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generationcited by 58
  9. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  10. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  11. 2023Parallelizing non-linear sequential models over the sequence lengthcited by 45
  12. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
38 more
  1. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 41
  2. 2025MagCache: Fast Video Generation with Magnitude-Aware Cachecited by 41
  3. 2024Towards Scalable and Stable Parallelization of Nonlinear RNNscited by 38
  4. 2024Diffusion Model Predictive Controlcited by 36
  5. 2024Reward Guided Latent Consistency Distillationcited by 35
  6. 2025Efficient Diffusion Models: A Surveycited by 35
  7. 2024T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitchingcited by 33
  8. 2023Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Modelscited by 31
  9. 2025Accelerated Diffusion Models via Speculative Samplingcited by 31
  10. 2024Accelerating Parallel Sampling of Diffusion Modelscited by 30
  11. 2024Video-Infinity: Distributed Long Video Generationcited by 30
  12. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 30
  13. 2024Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallelcited by 29
  14. 2025Distributional Diffusion Models with Scoring Rulescited by 25
  15. 2024AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoisingcited by 22
  16. 2023DreamPropeller: Supercharge Text-to-3D Generation with Parallel Samplingcited by 20
  17. 2025FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusioncited by 19
  18. 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
  19. 2025CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Modelscited by 18
  20. 2023TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Predictioncited by 17
  21. 2025Cached Adaptive Token Merging: Dynamic Token Reduction and Redundant Computation Elimination in Diffusion Modelcited by 16
  22. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  23. 2025Falcon: Fast Visuomotor Policies via Partial Denoisingcited by 15
  24. 2025ParaRNN: Unlocking Parallel Training of Nonlinear RNNs for Large Language Modelscited by 15
  25. 2024Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategycited by 14
  26. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  27. 2025Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Modelscited by 13
  28. 2025Generalised Flow Maps for Few-Step Generative Modelling on Riemannian Manifoldscited by 12
  29. 2024Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributionscited by 11
  30. 2025Diffusion Models are Secretly Exchangeable: Parallelizing DDPMs via Autospeculationcited by 11
  31. 2025Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generationcited by 11
  32. 2024SparseDiT: Token Sparsification for Efficient Diffusion Transformercited by 10
  33. 2025Learning to Integrate Diffusion ODEs by Averaging the Derivativescited by 10
  34. 2024Equivariant Denoisers Cannot Copy Graphs: Align Your Graph Diffusion Modelscited by 8
  35. 2024Parallel Sampling via Countingcited by 8
  36. 2025DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacecited by 8
  37. 2024Self-Refining Diffusion Samplers: Enabling Parallelization via Parareal Iterationscited by 7
  38. 2025Predictability Enables Parallelization of Nonlinear State Space Modelscited by 7

50 most cited of 112 All 112 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025It Takes Two to Tango: Two Parallel Samplers Improve Quality in Diffusion Models for Limited Steps
  2. 2026DRiffusion: Draft-and-Refine Process Parallelizes Diffusion Models with Ease
  3. 2025Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism
  4. 2025Pretrained Diffusion Models Are Inherently Skipped-Step Samplers
  5. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
  6. 2024DuoDiff: Accelerating Diffusion Models with a Dual-Backbone Approach
  7. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
  8. 2024One Step Diffusion via Shortcut Models