# Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance

arXiv: https://arxiv.org/abs/2212.06359 · HTML: https://mlsanity.com/paper/2212.06359/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 18 of its 50 references that are in the map.
- First posted on arXiv 13 Dec 2022.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis](https://mlsanity.com/paper/2203.13508/) (2022)

## Built on by (most cited first, 30 of 50)

- [Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models](https://mlsanity.com/paper/2306.09251/) (2023)
- [Optimizing DDPM Sampling with Shortcut Fine-Tuning](https://mlsanity.com/paper/2301.13362/) (2023)
- [A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models](https://mlsanity.com/paper/2408.02320/) (2024)
- [Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial](https://mlsanity.com/paper/2402.07487/) (2024)
- [Convergence Analysis of Probability Flow ODE for Score-based Generative Models](https://mlsanity.com/paper/2404.09730/) (2024)
- [DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises](https://mlsanity.com/paper/2302.10025/) (2023)
- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Closed-Form Diffusion Models](https://mlsanity.com/paper/2310.12395/) (2023)
- [A Good Score Does not Lead to A Good Generative Model](https://mlsanity.com/paper/2401.04856/) (2024)
- [Contractive Diffusion Probabilistic Models](https://mlsanity.com/paper/2401.13115/) (2024)
- [Beyond Log-Concavity and Score Regularity: Improved Convergence Bounds for Score-Based Generative Models in W2-distance](https://mlsanity.com/paper/2501.02298/) (2025)
- [Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey](https://mlsanity.com/paper/2305.05080/) (2023)
- [On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates](https://mlsanity.com/paper/2311.13584/) (2023)
- [Improved Convergence of Score-Based Diffusion Models via Prediction-Correction](https://mlsanity.com/paper/2305.14164/) (2023)
- [Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective](https://mlsanity.com/paper/2405.16418/) (2024)
- [Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance](https://mlsanity.com/paper/2508.03210/) (2025)
- [Systematic Assessment of Tabular Data Synthesis](https://mlsanity.com/paper/2402.06806/) (2024)
- [Score-based generative models are provably robust: an uncertainty quantification perspective](https://mlsanity.com/paper/2405.15754/) (2024)
- [Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients](https://mlsanity.com/paper/2505.03432/) (2025)
- [Correlated and Multi-frequency Diffusion Modeling for Highly Under-sampled MRI Reconstruction](https://mlsanity.com/paper/2309.00853/) (2023)
- [Wasserstein proximal operators describe score-based generative models and resolve memorization](https://mlsanity.com/paper/2402.06162/) (2024)
- [Optimal Transport-Guided Conditional Score-Based Diffusion Models](https://mlsanity.com/paper/2311.01226/) (2023)
- [On the Convergence and Straightness of Rectified Flow](https://mlsanity.com/paper/2410.14949/) (2024)
- [A Note on the Convergence of Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2312.05989/) (2023)
- [Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Bounds](https://mlsanity.com/paper/2506.09681/) (2025)
- [Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss](https://mlsanity.com/paper/2602.07022/) (2026)
- [CoInD: Enabling Logical Compositions in Diffusion Models](https://mlsanity.com/paper/2503.01145/) (2025)
- [Neural Entropy](https://mlsanity.com/paper/2409.03817/) (2024)
- [Generating Physical Dynamics under Priors](https://mlsanity.com/paper/2409.00730/) (2024)
- [Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport](https://mlsanity.com/paper/2502.12631/) (2025)

## Abstract (arXiv)

Score-based generative models are shown to achieve remarkable empirical performances in various applications such as image generation and audio synthesis. However, a theoretical understanding of score-based diffusion models is still incomplete. Recently, Song et al. showed that the training objective of score-based generative models is equivalent to minimizing the Kullback-Leibler divergence of the generated distribution from the data distribution. In this work, we show that score-based models also minimize the Wasserstein distance between them under suitable assumptions on the model. Specifically, we prove that the Wasserstein distance is upper bounded by the square root of the objective function up to multiplicative constants and a fixed constant offset. Our proof is based on a novel application of the theory of optimal transport, which can be of independent interest to the society. Our numerical experiments support our findings. By analyzing our upper bounds, we provide a few techniques to obtain tighter upper bounds.