mlsanityData to 7 Oct 2026

Optimal score estimation via empirical Bayes smoothing

arXiv 2402.07747 ↗First posted 12 Feb 2024math.ST stat.ML stat.THcited by 60

Not on the map. Why?

Cited by 48 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study the problem of estimating the score function of an unknown probability distribution ρ^* from n independent and identically distributed observations in d dimensions. Assuming that ρ^* is subgaussian and has a Lipschitz-continuous score function s^*, we establish the optimal rate of tilde Theta(n^-frac2d+4) for this estimation problem under the loss function |hat s - s^*|2L2(ρ^*) that is commonly used in the score matching literature, highlighting the curse of dimensionality where sample complexity for accurate score estimation grows exponentially with the dimension d. Leveraging key insights in empirical Bayes theory as well as a new convergence rate of smoothed empirical distribution in Hellinger distance, we show that a regularized score estimator based on a Gaussian kernel attains this rate, shown optimal by a matching minimax lower bound. We also discuss extensions to estimating β-H"older continuous scores with β ≤ 1, as well as the implication of our theory on the sample complexity of score-based generative models.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 33 of its 56 references that are in the map. First posted on arXiv 12 Feb 2024.

Builds on

33 of 56 references.

References · 33

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  5. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  6. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  7. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  8. 2021How to Train Your Energy-Based Modelscited by 271
  9. 2022Convergence of denoising diffusion models under the manifold hypothesiscited by 237
  10. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localizationcited by 230
  11. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
  12. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptionscited by 215
21 more
  1. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  2. 2022Convergence of score-based generative modeling for general data distributionscited by 179
  3. 2022Convergence for score-based generative modeling with polynomial complexitycited by 176
  4. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 171
  5. 2023Diffusion Models are Minimax Optimal Distribution Estimatorscited by 159
  6. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 144
  7. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 126
  8. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  9. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  10. 2017Convergence of Langevin MCMC in KL-divergencecited by 86
  11. 2023Closed-Form Diffusion Modelscited by 46
  12. 2013Density Estimation in Infinite Dimensional Exponential Familiescited by 41
  13. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  14. 2018Coupling and Convergence for Hamiltonian Monte Carlocited by 36
  15. 2024A Good Score Does not Lead to A Good Generative Modelcited by 36
  16. 2018Deep Energy Estimator Networkscited by 34
  17. 2020Nonparametric Score Estimatorscited by 22
  18. 2017On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoisingcited by 19
  19. 2014Two Modeling Strategies for Empirical Bayes Estimationcited by 16
  20. 2021Sharp regret bounds for empirical Bayes and compound decision problemscited by 8
  21. 2022Empirical Bayes estimation: When does g-modeling beat f-modeling in theory (and in practice)?cited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 24; 2026: 16.

60 papers.

Sort:

By area:

  1. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  2. 2024Gradient Guidance for Diffusion Models: An Optimization Perspectivecited by 59
  3. 2024Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theorycited by 58
  4. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  5. 2024Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptionscited by 45
  6. 2024Learning general Gaussian mixtures with efficient score matchingcited by 43
  7. 2024Linear Convergence of Diffusion Models Under the Manifold Hypothesiscited by 42
  8. 2024Learning Mixtures of Gaussians Using Diffusion Modelscited by 41
  9. 2024On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)cited by 39
  10. 2025Memorization and Regularization in Generative Diffusion Modelscited by 39
  11. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
  12. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
38 more
  1. 2024From optimal score matching to optimal samplingcited by 27
  2. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  3. 2024Evaluating the design space of diffusion-based generative modelscited by 25
  4. 2024Diffusion Model for Data-Driven Black-Box Optimizationcited by 24
  5. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  6. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  7. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  8. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approachcited by 19
  9. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  10. 2025On the Interpolation Effect of Score Smoothing in Diffusion Modelscited by 15
  11. 2025A solvable model of learning generative diffusion: theory and insightscited by 14
  12. 2024Shallow diffusion networks provably learn hidden low-dimensional structurecited by 12
  13. 2025Mixing Time of the Proximal Sampler in Relative Fisher Information via Strong Data Processing Inequalitycited by 11
  14. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  15. 2024Plug-in estimation of Schr"odinger bridgescited by 8
  16. 2025Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Boundscited by 8
  17. 2025Non-asymptotic error bounds for probability flow ODEs under weak log-concavitycited by 7
  18. 2025Distinguishing Cause from Effect with Causal Velocity Modelscited by 6
  19. 2025DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximizationcited by 6
  20. 2026Training-Free Adaptation of Diffusion Models via Doob's h-Transformcited by 6
  21. 2025Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learningcited by 5
  22. 2025Localized Diffusion Modelscited by 5
  23. 2025Algorithm- and Data-Dependent Generalization Bounds for Diffusion Modelscited by 4
  24. 2024Cryptographic Hardness of Score Estimationcited by 3
  25. 2025On the Contractivity of Stochastic Interpolation Flowcited by 3
  26. 2025Kernel-Smoothed Scores for Denoising Diffusion: A Bias-Variance Studycited by 3
  27. 2026Quantifying Error Propagation and Model Collapse in Diffusion Modelscited by 3
  28. 2024Differentially Private Kernel Density Estimationcited by 2
  29. 2024Stein's method for marginals on large graphical modelscited by 2
  30. 2025Regularization can make diffusion models more efficientcited by 2
  31. 2025Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Accesscited by 2
  32. 2025Convergence Dynamics of Over-Parameterized Score Matching for a Single Gaussiancited by 2
  33. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2
  34. 2025Data Generation without Function Estimationcited by 1
  35. 2025Non-asymptotic convergence bound of conditional diffusion modelscited by 1
  36. 2026Provably Reliable Classifier Guidance via Cross-Entropy Controlcited by 1
  37. 2026Load--Reserve Wasserstein Propagation for Isotropic Diffusion Samplerscited by 1
  38. 2026On the Robustness of Distribution Support under Diffusion Guidancecited by 1

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning the score under shape constraints
  2. 2025Heteroscedasticity of Denoising Score Matching with Generalised Smooth Noise
  3. 2025Implicit score matching meets denoising score matching: improved rates of convergence and log-density Hessian estimation
  4. 2026Minimax Optimality of Score-Entropy Discrete Diffusion
  5. 2026Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees
  6. 2026Robustness of Diffusion Models under Distribution Shift
  7. 2025Sampling Binary Data by Denoising through Score Functions
  8. 2026Sobolev Regularized Score Difference Estimation in Diffusion Models