mlsanityData to 7 Oct 2026

A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models

arXiv 2408.02320 ↗First posted 5 Aug 2024cs.LG cs.NA eess.SPcited by 64

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models, which convert noise into new data instances by learning to reverse a diffusion process, have become a cornerstone in contemporary generative modeling. In this work, we develop non-asymptotic convergence theory for a popular diffusion-based sampler (i.e., the probability flow ODE sampler) in discrete time, assuming access to ℓ2-accurate estimates of the (Stein) score functions. For distributions in ℝd, we prove that d/varepsilon iterations -- modulo some logarithmic and lower-order terms -- are sufficient to approximate the target distribution to within varepsilon total-variation distance. This is the first result establishing nearly linear dimension-dependency (in d) for the probability flow ODE sampler. Imposing only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), our results also characterize how ℓ2 score estimation errors affect the quality of the data generation processes. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach without the need of resorting to SDE and ODE toolboxes.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 70 of its 87 references that are in the map. First posted on arXiv 5 Aug 2024.

Builds on

70 of 87 references.

Closest ancestors · 50

Same direction, newest first

  1. 2024Provable Statistical Rates for Consistency Diffusion Models23 Jun
  2. 2024Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallel3 Jun
  3. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity24 May
  4. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models23 May
  5. 2024Learning general Gaussian mixtures with efficient score matching29 Apr
  6. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Models15 Apr
  7. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization11 Apr
  8. 2024Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory18 Mar
  9. 2024Accelerating Convergence of Score-Based Diffusion Models, Provably6 Mar
  10. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models3 Mar
  11. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion models3 Mar
  12. 2024Dynamical Regimes of Diffusion Models28 Feb
38 more
  1. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach21 Feb
  2. 2024Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization12 Feb
  3. 2024Towards a mathematical theory for consistency training in diffusion models12 Feb
  4. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial12 Feb
  5. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances31 Jan
  6. 2024Contractive Diffusion Probabilistic Models23 Jan
  7. 2023A Note on the Convergence of Denoising Diffusion Probabilistic Models10 Dec
  8. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  9. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  10. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  11. 2023Error Bounds for Flow Matching Methods26 May
  12. 2023Improved Convergence of Score-Based Diffusion Models via Prediction-Correction23 May
  13. 2023The probability flow ODE is provably fast19 May
  14. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  15. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  16. 2023Consistency Models2 Mar
  17. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  18. 2023Geometry of Score Based Generative Models9 Feb
  19. 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance13 Dec
  20. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  21. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  22. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  23. 2022Diffusion Models in Vision: A Survey10 Sep
  24. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  25. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models31 Aug
  26. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  27. 2022Classifier-Free Diffusion Guidance26 Jul
  28. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  29. 2022Score-Based Generative Models Detect Manifolds2 Jun
  30. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  31. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  32. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  33. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  34. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  35. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  36. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  37. 2020Denoising Diffusion Implicit Models6 Oct
  38. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 20

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  6. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  7. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  8. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  9. 2020Adversarial score matching and improved sampling for image generationcited by 95
  10. 2024Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstructioncited by 73
  11. 2022Statistical Efficiency of Score Matching: The View from Isoperimetrycited by 63
  12. 2022Sampling from the Sherrington-Kirkpatrick Gibbs measure via algorithmic stochastic localizationcited by 48
8 more
  1. 2022Normalizing flow neural networks by JKO schemecited by 39
  2. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  3. 2023A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Modelscited by 37
  4. 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
  5. 2024Convergence of Continuous Normalizing Flows for Learning Probability Distributionscited by 24
  6. 2022A Non-Asymptotic Framework for Approximate Message Passing in Spiked Modelscited by 18
  7. 2024Optimal convex M-estimation via score matchingcited by 9
  8. 2024Provably Efficient Posterior Sampling for Sparse Linear Regression via Measure Decompositioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 32; 2026: 16.

64 papers.

Sort:

By area:

  1. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  2. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptionscited by 48
  3. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  4. 2024What does guidance do? A fine-grained analysis in a simple settingcited by 45
  5. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionalitycited by 44
  6. 2024Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structurecited by 38
  7. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
  8. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
  9. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  10. 2025Optimal Inference Schedules for Masked Diffusion Modelscited by 25
  11. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysiscited by 24
  12. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
38 more
  1. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  2. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  3. 2024Instance-dependent Convergence Theory for Diffusion Modelscited by 21
  4. 2024On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimalitycited by 21
  5. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  6. 2025Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Accelerationcited by 17
  7. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  8. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  9. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  10. 2024Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplerscited by 14
  11. 2026High-accuracy sampling for diffusion models and log-concave distributionscited by 13
  12. 2024Shallow diffusion networks provably learn hidden low-dimensional structurecited by 12
  13. 2025Generalization of Diffusion Models Arises with a Balanced Representation Spacecited by 10
  14. 2025Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Modelscited by 9
  15. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  16. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Modelscited by 8
  17. 2025Provable Efficiency of Guidance in Diffusion Models for General Data Distributioncited by 8
  18. 2025A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptionscited by 8
  19. 2025Sublinear iterations can suffice even for DDPMscited by 7
  20. 2025Towards a unified framework for guided diffusion modelscited by 7
  21. 2026High-accuracy and dimension-free sampling with diffusionscited by 7
  22. 2025Longitudinal Flow Matching for Trajectory Modelingcited by 6
  23. 2024Local Flow Matching Generative Modelscited by 5
  24. 2025Distribution estimation via Flow Matching with Lipschitz guaranteescited by 5
  25. 2025Parallel Sampling via Autospeculationcited by 5
  26. 2025When and how can inexact generative models still sample from the data manifold?cited by 4
  27. 2025Fast Score-Based Sampling via Log-Concave Reductionscited by 4
  28. 2026Provable diffusion-based posterior sampling for linear inverse problems via DDIMcited by 4
  29. 2025Beyond Scores: Proximal Diffusion Modelscited by 3
  30. 2025Elucidating Rectified Flow with Deterministic Sampler: Polynomial Discretization Complexity for Multi and One-step Modelscited by 3
  31. 2025Dimension-free error estimate for diffusion model and optimal schedulingcited by 3
  32. 2025Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networkscited by 3
  33. 2024Differentially Private Kernel Density Estimationcited by 2
  34. 2025Multi-Step Consistency Models: Fast Generation with Theoretical Guaranteescited by 2
  35. 2025Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Accesscited by 2
  36. 2026Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspectivecited by 2
  37. 2026Total Variation Rates for Riemannian Flow Matchingcited by 2
  38. 2026Query Lower Bounds for Diffusion Samplingcited by 2

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Probability flow ODEs in score-based and reflected diffusion models
  2. 2025Information-Theoretic Proofs for Diffusion Sampling
  3. 2025Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance
  4. 2025Non-asymptotic error bounds for probability flow ODEs under weak log-concavity
  5. 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
  6. 2025Diffusion Models: A Mathematical Introduction
  7. 2024New algorithms for sampling and diffusion models
  8. 2023A Geometric Perspective on Diffusion Models