mlsanityData to 7 Oct 2026

Accelerating Convergence of Score-Based Diffusion Models, Provably

arXiv 2403.03852 ↗First posted 6 Mar 2024cs.LG cs.AI cs.ITcited by 74

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Score-based diffusion models, while achieving remarkable empirical performance, often suffer from low sampling speed, due to extensive function evaluations needed during the sampling phase. Despite a flurry of recent activities towards speeding up diffusion generative modeling in practice, theoretical underpinnings for acceleration techniques remain severely limited. In this paper, we design novel training-free algorithms to accelerate popular deterministic (i.e., DDIM) and stochastic (i.e., DDPM) samplers. Our accelerated deterministic sampler converges at a rate O(1/T2) with T the number of steps, improving upon the O(1/T) rate for the DDIM sampler; and our accelerated stochastic sampler converges at a rate O(1/T), outperforming the rate O(1/√T) for the DDPM sampler. The design of our algorithms leverages insights from higher-order approximation, and shares similar intuitions as popular high-order ODE solvers like the DPM-Solver-2. Our theory accommodates ℓ2-accurate score estimates, and does not require log-concavity or smoothness on the target distribution.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 47 of its 57 references that are in the map. First posted on arXiv 6 Mar 2024.

Builds on

47 of 57 references.

Closest ancestors · 40

Same direction, newest first

  1. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models3 Mar
  2. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach21 Feb
  3. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization12 Feb
  4. 2024Towards a mathematical theory for consistency training in diffusion models12 Feb
  5. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial12 Feb
  6. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances31 Jan
  7. 2024Contractive Diffusion Probabilistic Models23 Jan
  8. 2024A Good Score Does not Lead to A Good Generative Model10 Jan
  9. 2023Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models18 Nov
  10. 2023SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models10 Sep
  11. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  12. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
28 more
  1. 2023Error Bounds for Flow Matching Methods26 May
  2. 2023The probability flow ODE is provably fast19 May
  3. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  4. 2023Consistency Models2 Mar
  5. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
  6. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  7. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  8. 2022On Distillation of Guided Diffusion Models6 Oct
  9. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  10. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  11. 2022Diffusion Models in Vision: A Survey10 Sep
  12. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  13. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models31 Aug
  14. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  15. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  16. 2022gDDIM: Generalized denoising diffusion implicit models11 Jun
  17. 2022Score-Based Generative Models Detect Manifolds2 Jun
  18. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  19. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  20. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  21. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  22. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  23. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  24. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  25. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  26. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  27. 2020Denoising Diffusion Implicit Models6 Oct
  28. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 7

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  5. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  6. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  7. 2020Efficient Learning of Generative Models via Finite-Difference Score Matchingcited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 39; 2026: 14.

74 papers.

Sort:

By area:

  1. 2025How to build a consistency model: Learning flow maps via self-distillationcited by 96
  2. 2024Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstructioncited by 73
  3. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  4. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 64
  5. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Modelscited by 56
  6. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptionscited by 48
  7. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  8. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  9. 2024What does guidance do? A fine-grained analysis in a simple settingcited by 45
  10. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionalitycited by 44
  11. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  12. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 40
38 more
  1. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
  2. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
  3. 2024Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallelcited by 29
  4. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  5. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysiscited by 24
  6. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  7. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  8. 2024Instance-dependent Convergence Theory for Diffusion Modelscited by 21
  9. 2024On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimalitycited by 21
  10. 2025Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distancecited by 20
  11. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  12. 2024Constrained Diffusion Models via Dual Trainingcited by 19
  13. 2025Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approachcited by 18
  14. 2025Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Accelerationcited by 17
  15. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  16. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  17. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  18. 2025Lipschitz-Guided Design of Interpolation Schedules in Generative Modelscited by 15
  19. 2024Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplerscited by 14
  20. 2026High-accuracy sampling for diffusion models and log-concave distributionscited by 13
  21. 2025Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Modelscited by 9
  22. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  23. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Modelscited by 8
  24. 2025Provable Efficiency of Guidance in Diffusion Models for General Data Distributioncited by 8
  25. 2025Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Boundscited by 8
  26. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
  27. 2026Sharp Convergence Rates for Masked Diffusion Modelscited by 8
  28. 2025Sublinear iterations can suffice even for DDPMscited by 7
  29. 2026High-accuracy and dimension-free sampling with diffusionscited by 7
  30. 2024Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplerscited by 6
  31. 2025Information-Theoretic Proofs for Diffusion Samplingcited by 6
  32. 2024Diffusion Transformer Captures Spatial-Temporal Dependencies: A Theory for Gaussian Process Datacited by 5
  33. 2025Finite-Time Analysis of Discrete-Time Stochastic Interpolantscited by 4
  34. 2025Improving Diffusion-based Inverse Algorithms under Few-Step Constraint via Learnable Linear Extrapolationcited by 4
  35. 2025Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptionscited by 4
  36. 2025Pathway to O(√d) Complexity bound under Wasserstein metric of flow-based modelscited by 4
  37. 2024Model Free Prediction with Uncertainty Assessmentcited by 3
  38. 2024Leveraging Multimodal Diffusion Models to Accelerate Imaging with Side Informationcited by 3

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
  2. 2026From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models
  3. 2025SADA: Stability-guided Adaptive Diffusion Acceleration
  4. 2023On Accelerating Diffusion-Based Sampling Process via Improved Integration Approximation
  5. 2026Formalizing the Sampling Design Space of Diffusion-Based Generative Models via Adaptive Solvers and Wasserstein-Bounded Timesteps
  6. 2023Improved Convergence of Score-Based Diffusion Models via Prediction-Correction
  7. 2022Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling
  8. 2025Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling