mlsanityData to 7 Oct 2026

Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance

arXiv 2212.06359 ↗First posted 13 Dec 2022cs.LG cs.AI cs.NAcited by 50

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Score-based generative models are shown to achieve remarkable empirical performances in various applications such as image generation and audio synthesis. However, a theoretical understanding of score-based diffusion models is still incomplete. Recently, Song et al. showed that the training objective of score-based generative models is equivalent to minimizing the Kullback-Leibler divergence of the generated distribution from the data distribution. In this work, we show that score-based models also minimize the Wasserstein distance between them under suitable assumptions on the model. Specifically, we prove that the Wasserstein distance is upper bounded by the square root of the objective function up to multiplicative constants and a fixed constant offset. Our proof is based on a novel application of the theory of optimal transport, which can be of independent interest to the society. Our numerical experiments support our findings. By analyzing our upper bounds, we provide a few techniques to obtain tighter upper bounds.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 18 of its 50 references that are in the map. First posted on arXiv 13 Dec 2022.

Builds on

18 of 50 references.

Closest ancestors · 7

Same direction, newest first

  1. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  2. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  3. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  5. 2020Denoising Diffusion Implicit Models6 Oct
  6. 2020Denoising Diffusion Probabilistic Models19 Jun
  7. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 11

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  5. 2017Decoupled Weight Decay Regularizationcited by 1,722
  6. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  7. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  8. 2018Invertible Residual Networkscited by 364
  9. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  10. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  11. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesiscited by 64

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 19; 2025: 13; 2026: 5.

50 papers.

Sort:

By area:

  1. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Modelscited by 84
  2. 2023Optimizing DDPM Sampling with Shortcut Fine-Tuningcited by 81
  3. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 64
  4. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
  5. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Modelscited by 56
  6. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  7. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  8. 2023Closed-Form Diffusion Modelscited by 46
  9. 2024A Good Score Does not Lead to A Good Generative Modelcited by 36
  10. 2024Contractive Diffusion Probabilistic Modelscited by 29
  11. 2023Scalable Optimal Transport Methods in Machine Learning: A Contemporary Surveycited by 27
  12. 2025Beyond Log-Concavity and Score Regularity: Improved Convergence Bounds for Score-Based Generative Models in W2-distancecited by 27
38 more
  1. 2023On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimatescited by 24
  2. 2024Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspectivecited by 24
  3. 2023Improved Convergence of Score-Based Diffusion Models via Prediction-Correctioncited by 23
  4. 2025Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distancecited by 20
  5. 2024Systematic Assessment of Tabular Data Synthesiscited by 17
  6. 2025Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradientscited by 15
  7. 2023Correlated and Multi-frequency Diffusion Modeling for Highly Under-sampled MRI Reconstructioncited by 14
  8. 2024Wasserstein proximal operators describe score-based generative models and resolve memorizationcited by 14
  9. 2024Score-based generative models are provably robust: an uncertainty quantification perspectivecited by 14
  10. 2023Optimal Transport-Guided Conditional Score-Based Diffusion Modelscited by 11
  11. 2024On the Convergence and Straightness of Rectified Flowcited by 11
  12. 2023A Note on the Convergence of Denoising Diffusion Probabilistic Modelscited by 9
  13. 2026Condition Errors Refinement in Autoregressive Image Generation with Diffusion Losscited by 9
  14. 2025CoInD: Enabling Logical Compositions in Diffusion Modelscited by 8
  15. 2025Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Boundscited by 8
  16. 2024Neural Entropycited by 7
  17. 2024Generating Physical Dynamics under Priorscited by 6
  18. 2025Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transportcited by 6
  19. 2025Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibilitycited by 4
  20. 2023Straighter Flow Matching via a Diffusion-Based Coupling Priorcited by 3
  21. 2024Speed-accuracy relations for diffusion models: Wisdom from nonequilibrium thermodynamics and optimal transportcited by 2
  22. 2024HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative modelscited by 2
  23. 2024Solving Prior Distribution Mismatch in Diffusion Models via Optimal Transportcited by 2
  24. 2025Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Mapscited by 2
  25. 2024From Function to Distribution Modeling: A PAC-Generative Approach to Offline Optimizationcited by 1
  26. 2024Learning Quantized Adaptive Conditions for Diffusion Modelscited by 1
  27. 2025Non-asymptotic convergence bound of conditional diffusion modelscited by 1
  28. 2026Wasserstein Convergence of ODE-Based Samplers in Decentralized Diffusion Model via Velocity Field Decompositioncited by 1
  29. 2023Deep Generative Modeling with Backward Stochastic Differential Equationscited by 0
  30. 2023Generalized Contrastive Divergence: Joint Training of Energy-Based Model and Diffusion Model through Inverse Reinforcement Learningcited by 0
  31. 2024Zero-shot Dynamic MRI Reconstruction with Global-to-local Diffusion Modelcited by 0
  32. 2025Mean and Variance Estimation Complexity in Arbitrary Distributions via Wasserstein Minimizationcited by 0
  33. 2025Compositional World Knowledge leads to High Utility Synthetic datacited by 0
  34. 2025Jet Image Generation in High Energy Physics Using Diffusion Modelscited by 0
  35. 2025Thermodynamic Performance Limits for Score-Based Diffusion Modelscited by 0
  36. 2026Generative Neural Operators through Diffusion Last Layercited by 0
  37. 2026Stage-wise Distortion-Perception Traversal in Zero-shot Inverse Problems with Diffusion Modelscited by 0
  38. 2026Universal Local Error and Realized Amplification for the First-Order EDM Predictorcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Geometry of Score Based Generative Models
  2. 2022Convergence of score-based generative modeling for general data distributions
  3. 2023Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models
  4. 2026Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data
  5. 2024Wasserstein Bounds for generative diffusion models with Gaussian tail targets
  6. 2024An analysis of the noise schedule for score-based generative models
  7. 2025Non-asymptotic error bounds for probability flow ODEs under weak log-concavity
  8. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions