# From optimal score matching to optimal sampling

arXiv: https://arxiv.org/abs/2409.07032 · HTML: https://mlsanity.com/paper/2409.07032/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 11 Sep 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 20 of 20)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Convergence of score-based generative modeling for general data distributions](https://mlsanity.com/paper/2209.12381/) (2022)
- [Convergence for score-based generative modeling with polynomial complexity](https://mlsanity.com/paper/2206.06227/) (2022)
- [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023)
- [Generative Modeling with Denoising Auto-Encoders and Langevin Sampling](https://mlsanity.com/paper/2002.00107/) (2020)
- [An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization](https://mlsanity.com/paper/2404.07771/) (2024)
- [Optimal score estimation via empirical Bayes smoothing](https://mlsanity.com/paper/2402.07747/) (2024)
- [Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial](https://mlsanity.com/paper/2402.07487/) (2024)
- [Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions](https://mlsanity.com/paper/2402.15602/) (2024)
- [How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric View](https://mlsanity.com/paper/1712.08244/) (2017)
- [Sharp regret bounds for empirical Bayes and compound decision problems](https://mlsanity.com/paper/2109.03943/) (2021)

## Built on by (most cited first, 28 of 28)

- [Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality](https://mlsanity.com/paper/2410.18784/) (2024)
- [Low-dimensional adaptation of diffusion models: Convergence in total variation](https://mlsanity.com/paper/2501.12982/) (2025)
- [Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models](https://mlsanity.com/paper/2410.04760/) (2024)
- [Provable Acceleration for Diffusion Models under Minimal Assumptions](https://mlsanity.com/paper/2410.23285/) (2024)
- [Minimax Optimality of the Probability Flow ODE for Diffusion Models](https://mlsanity.com/paper/2503.09583/) (2025)
- [Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures](https://mlsanity.com/paper/2504.05300/) (2025)
- [Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models](https://mlsanity.com/paper/2505.21400/) (2025)
- [On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimality](https://mlsanity.com/paper/2411.17522/) (2024)
- [FunDiff: Diffusion Models over Function Spaces for Physics-Informed Generative Modeling](https://mlsanity.com/paper/2506.07902/) (2025)
- [A solvable model of learning generative diffusion: theory and insights](https://mlsanity.com/paper/2501.03937/) (2025)
- [Shallow diffusion networks provably learn hidden low-dimensional structure](https://mlsanity.com/paper/2410.11275/) (2024)
- [Non-asymptotic error bounds for probability flow ODEs under weak log-concavity](https://mlsanity.com/paper/2510.17608/) (2025)
- [Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity](https://mlsanity.com/paper/2603.20645/) (2026)
- [DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximization](https://mlsanity.com/paper/2510.12691/) (2025)
- [Do Large Language Models (Really) Need Statistical Foundations?](https://mlsanity.com/paper/2505.19145/) (2025)
- [Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees](https://mlsanity.com/paper/2601.06715/) (2026)
- [Quantifying Error Propagation and Model Collapse in Diffusion Models](https://mlsanity.com/paper/2602.16601/) (2026)
- [Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions](https://mlsanity.com/paper/2605.30153/) (2026)
- [Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regime](https://mlsanity.com/paper/2608.23938/) (2026)
- [Regularization can make diffusion models more efficient](https://mlsanity.com/paper/2502.09151/) (2025)
- [Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors](https://mlsanity.com/paper/2606.06179/) (2026)
- [Provably Reliable Classifier Guidance via Cross-Entropy Control](https://mlsanity.com/paper/2601.21200/) (2026)
- [Reflected diffusion models adapt to low-dimensional data](https://mlsanity.com/paper/2603.24495/) (2026)
- [On the Robustness of Distribution Support under Diffusion Guidance](https://mlsanity.com/paper/2605.07220/) (2026)
- [Diffusion Models for Adaptive Sequential Data Generation](https://mlsanity.com/paper/2606.06007/) (2026)
- [Computational and Statistical Guarantees of the textitc-Rectified flow](https://mlsanity.com/paper/2608.02487/) (2026)
- [Robustness of Diffusion Models under Distribution Shift](https://mlsanity.com/paper/2609.27546/) (2026)
- [A Stochastic Analysis Perspective on Diffusion Models](https://mlsanity.com/paper/2610.14133/) (2026)

## Abstract (arXiv)

The recent, impressive advances in algorithmic generation of high-fidelity image, audio, and video are largely due to great successes in score-based diffusion models. A key implementing step is score matching, that is, the estimation of the score function of the forward diffusion process from training data. As shown in earlier literature, the total variation distance between the law of a sample generated from the trained diffusion model and the ground truth distribution can be controlled by the score matching risk. Despite the widespread use of score-based diffusion models, basic theoretical questions concerning exact optimal statistical rates for score estimation and its application to density estimation remain open. We establish the sharp minimax rate of score estimation for smooth, compactly supported densities. Formally, given (n) i.i.d. samples from an unknown (α)-H"older density (f) supported on ([-1, 1]), we prove the minimax rate of estimating the score function of the diffused distribution (f * N(0, t)) with respect to the score matching loss is (frac1nt2 wedge frac1nt3/2 wedge (tα-1 + n-2(α-1)/(2α+1))) for all (α > 0) and (t ≥ 0). As a consequence, it is shown the law (hatf) of a sample generated from the diffusion model achieves the sharp minimax rate (bE(dTV(hatf, f)2) lesssim n-2α/(2α+1)) for all (α > 0) without any extraneous logarithmic terms which are prevalent in the literature, and without the need for early stopping which has been required for all existing procedures to the best of our knowledge.