mlsanityData to 7 Oct 2026

From optimal score matching to optimal sampling

arXiv 2409.07032 ↗First posted 11 Sep 2024stat.ML cs.LGcited by 27

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The recent, impressive advances in algorithmic generation of high-fidelity image, audio, and video are largely due to great successes in score-based diffusion models. A key implementing step is score matching, that is, the estimation of the score function of the forward diffusion process from training data. As shown in earlier literature, the total variation distance between the law of a sample generated from the trained diffusion model and the ground truth distribution can be controlled by the score matching risk. Despite the widespread use of score-based diffusion models, basic theoretical questions concerning exact optimal statistical rates for score estimation and its application to density estimation remain open. We establish the sharp minimax rate of score estimation for smooth, compactly supported densities. Formally, given (n) i.i.d. samples from an unknown (α)-H"older density (f) supported on ([-1, 1]), we prove the minimax rate of estimating the score function of the diffused distribution (f * N(0, t)) with respect to the score matching loss is (frac1nt2 wedge frac1nt3/2 wedge (tα-1 + n-2(α-1)/(2α+1))) for all (α > 0) and (t ≥ 0). As a consequence, it is shown the law (hatf) of a sample generated from the diffusion model achieves the sharp minimax rate (bE(dTV(hatf, f)2) lesssim n-2α/(2α+1)) for all (α > 0) without any extraneous logarithmic terms which are prevalent in the literature, and without the need for early stopping which has been required for all existing procedures to the best of our knowledge.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 20 of its 41 references that are in the map. First posted on arXiv 11 Sep 2024.

Builds on

20 of 41 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization11 Apr
  2. 2024Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions23 Feb
  3. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial12 Feb
  4. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  5. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  6. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  7. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  11. 2020Denoising Diffusion Probabilistic Models19 Jun
  12. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 8

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  4. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  5. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  6. 2024Optimal score estimation via empirical Bayes smoothingcited by 60
  7. 2017How Well Can Generative Adversarial Networks Learn Densities: A Nonparametric Viewcited by 14
  8. 2021Sharp regret bounds for empirical Bayes and compound decision problemscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 12.

27 papers.

Sort:

By area:

  1. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionalitycited by 44
  2. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
  3. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
  4. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  5. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  6. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  7. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  8. 2024On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimalitycited by 21
  9. 2025FunDiff: Diffusion Models over Function Spaces for Physics-Informed Generative Modelingcited by 19
  10. 2025A solvable model of learning generative diffusion: theory and insightscited by 14
  11. 2024Shallow diffusion networks provably learn hidden low-dimensional structurecited by 12
  12. 2025Non-asymptotic error bounds for probability flow ODEs under weak log-concavitycited by 7
15 more
  1. 2025DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximizationcited by 6
  2. 2026Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexitycited by 6
  3. 2025Do Large Language Models (Really) Need Statistical Foundations?cited by 5
  4. 2026Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guaranteescited by 4
  5. 2026Quantifying Error Propagation and Model Collapse in Diffusion Modelscited by 3
  6. 2026Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regimecited by 3
  7. 2025Regularization can make diffusion models more efficientcited by 2
  8. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2
  9. 2026Provably Reliable Classifier Guidance via Cross-Entropy Controlcited by 1
  10. 2026Reflected diffusion models adapt to low-dimensional datacited by 1
  11. 2026On the Robustness of Distribution Support under Diffusion Guidancecited by 1
  12. 2026Diffusion Models for Adaptive Sequential Data Generationcited by 1
  13. 2026Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errorscited by 1
  14. 2026Computational and Statistical Guarantees of the textitc-Rectified flowcited by 0
  15. 2026Robustness of Diffusion Models under Distribution Shiftcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data
  2. 2025DDPM Score Matching and Distribution Learning
  3. 2024Random Walks with Tweedie: A Unified View of Score-Based Diffusion Models
  4. 2026Minimax Optimality of Score-Entropy Discrete Diffusion
  5. 2025Optimizing Input of Denoising Score Matching is Biased Towards Higher Score Norm
  6. 2026Tightening the Score Matching Gap for Diffusion Models
  7. 2025From Score Matching to Diffusion: A Fine-Grained Error Analysis in the Gaussian Setting
  8. 2022From Denoising Diffusions to Denoising Markov Models