mlsanityData to 14 Oct 2026

Improved Convergence of Score-Based Diffusion Models via Prediction-Correction

arXiv 2305.14164 ↗First posted 23 May 2023cs.LG math.ST stat.MLcited by 24

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Score-based generative models (SGMs) are powerful tools to sample from complex data distributions. Their underlying idea is to (i) run a forward process for time T1 by adding noise to the data, (ii) estimate its score function, and (iii) use such estimate to run a reverse process. As the reverse process is initialized with the stationary distribution of the forward one, the existing analysis paradigm requires T1→∞. This is however problematic: from a theoretical viewpoint, for a given precision of the score approximation, the convergence guarantee fails as T1 diverges; from a practical viewpoint, a large T1 increases computational costs and leads to error propagation. This paper addresses the issue by considering a version of the popular predictor-corrector scheme: after running the forward process, we first estimate the final distribution via an inexact Langevin dynamics and then revert the process. Our key technical contribution is to provide convergence guarantees which require to run the forward process only for a fixed finite time T1. Our bounds exhibit a mild logarithmic dependence on the input dimension and the subgaussian norm of the target distribution, have minimal assumptions on the data, and require only to control the L2 loss on the score approximation, which is the quantity minimized in practice.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 51 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

33 of 51 references.

Closest ancestors · 21

Same direction, newest first

  1. 2023The probability flow ODE is provably fast19 May
  2. 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance13 Dec
  3. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  4. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  5. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  6. 2022Diffusion Models in Vision: A Survey10 Sep
  7. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  8. 2022Applying Regularized Schr"odinger-Bridge-Based Stochastic Process in Generative Modeling15 Aug
  9. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  10. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  11. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Models10 Jun
  12. 2022Score-Based Generative Models Detect Manifolds2 Jun
9 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  2. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  3. 2022Conditional Simulation Using Diffusion Schr"odinger Bridges27 Feb
  4. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Probabilistic Models19 Jun
  9. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 12

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  4. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  5. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  6. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  7. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 227
  8. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 219
  9. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 119
  10. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  11. 2019A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Normcited by 106
  12. 2023Sampling, Diffusions, and Stochastic Localizationcited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 11; 2025: 6; 2026: 3.

24 papers.

Sort:

By area:

  1. 2024Accelerating Diffusion Sampling with Optimized Time Stepscited by 67
  2. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 65
  3. 2023KL Convergence Guarantees for Score diffusion models under minimal data assumptionscited by 63
  4. 2024How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Frameworkcited by 48
  5. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  6. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 39
  7. 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
  8. 2024Convergence of Continuous Normalizing Flows for Learning Probability Distributionscited by 26
  9. 2023On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimatescited by 25
  10. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  11. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approachcited by 19
  12. 2025Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradientscited by 15
12 more
  1. 2024Theoretical guarantees in KL for Diffusion Flow Matchingcited by 14
  2. 2023Closing the ODE-SDE gap in score-based diffusion models through the Fokker-Planck equationcited by 12
  3. 2024On the Convergence and Straightness of Rectified Flowcited by 11
  4. 2024On the Asymptotic Mean Square Error Optimality of Diffusion Modelscited by 10
  5. 2024Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplerscited by 7
  6. 2024Parallel Simulation for Log-concave Sampling and Score-based Diffusion Modelscited by 6
  7. 2025Dimension-free error estimate for diffusion model and optimal schedulingcited by 3
  8. 2025Solving Imaging Inverse Problems Using Plug-and-Play Denoisers: Regularization and Optimization Perspectivescited by 1
  9. 2026Diffusion Models for Adaptive Sequential Data Generationcited by 1
  10. 2025Concentration Inequalities for Stochastic Optimization of Unbounded Objective Functions with Application to Denoising Score Matchingcited by 0
  11. 2026Expressivity of Bi-Lipschitz Normalizing Flows: A Score-Based Diffusion Perspectivecited by 0
  12. 2026A Generalized McDiarmid Inequality: Sample Reuse and Non-Uniform Unbounded Differencescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Closed-Form Diffusion Models
  2. 2025Algorithm- and Data-Dependent Generalization Bounds for Diffusion Models
  3. 2026From Score Approximation to Distribution Approximation in Score-Based Diffusion Models
  4. 2023Preconditioned Score-based Generative Models
  5. 2026A Stochastic Analysis Perspective on Diffusion Models
  6. 2024Accelerating Convergence of Score-Based Diffusion Models, Provably
  7. 2026Quantifying Error Propagation and Model Collapse in Diffusion Models
  8. 2026Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data