mlsanityData to 13 Oct 2026

Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions

arXiv 2402.08082 ↗First posted 12 Feb 2024stat.ML cs.LGcited by 12

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

While score-based generative models (SGMs) have achieved remarkable success in enormous image generation tasks, their mathematical foundations are still limited. In this paper, we analyze the approximation and generalization of SGMs in learning a family of sub-Gaussian probability distributions. We introduce a notion of complexity for probability distributions in terms of their relative density with respect to the standard Gaussian measure. We prove that if the log-relative density can be locally approximated by a neural network whose parameters can be suitably bounded, then the distribution generated by empirical score matching approximates the target distribution in total variation with a dimension-independent rate. We illustrate our theory through examples, which include certain mixtures of Gaussians. An essential ingredient of our proof is to derive a dimension-free deep neural network approximation rate for the true score function associated with the forward process, which is interesting in its own right.

Key facts. Cited by 12 papers in the map (data through 13 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 46 references that are in the map. First posted on arXiv 12 Feb 2024.

Builds on

33 of 46 references.

Closest ancestors · 16

Same direction, newest first

  1. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  2. 2023The probability flow ODE is provably fast19 May
  3. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  4. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  5. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  6. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  7. 2022Diffusion Models in Vision: A Survey10 Sep
  8. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  9. 2022Understanding Diffusion Models: A Unified Perspective25 Aug
  10. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  11. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
4 more
  1. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
  2. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  3. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov

Other references · 17

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  2. 2016Error bounds for approximations with deep ReLU networkscited by 570
  3. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 496
  4. 2021Score-based diffusion models for accelerated MRIcited by 309
  5. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 280
  6. 2022Torsional Diffusion for Molecular Conformer Generationcited by 262
  7. 2016A vector-contraction inequality for Rademacher complexitiescited by 141
  8. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  9. 2020Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don'tcited by 95
  10. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  11. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  12. 2020Representation formulas and pointwise properties for Barron functionscited by 62
5 more
  1. 2020On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamicscited by 39
  2. 2017On the ability of neural nets to express distributionscited by 26
  3. 2021Interpreting diffusion score matching using normalizing flowcited by 9
  4. 2021On Energy-Based Models with Overparametrized Shallow Neural Networkscited by 8
  5. 2021Dual Training of Energy-Based Models with Overparametrized Shallow Neural Networkscited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 6.

12 papers.

  1. 2024Learning Mixtures of Gaussians Using Diffusion Modelscited by 42
  2. 2024Wasserstein Bounds for generative diffusion models with Gaussian tail targetscited by 10
  3. 2025Generalization of Diffusion Models Arises with a Balanced Representation Spacecited by 10
  4. 2024Dimension-independent rates for structured neural density estimationcited by 5
  5. 2026Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guaranteescited by 4
  6. 2024Breaking the curse of dimensionality in structured density estimationcited by 3
  7. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 3
  8. 2026Multi-Subspace Multi-Modal Modeling for Diffusion Models: Estimation, Convergence and Mixture of Expertscited by 1
  9. 2026Load--Reserve Wasserstein Propagation for Isotropic Diffusion Samplerscited by 1
  10. 2025Gaussian Process Tilted Nonparametric Density Estimation using Fisher Divergence Score Matchingcited by 0
  11. 2026Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Modelscited by 0
  12. 2026Robustness of Diffusion Models under Distribution Shiftcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Good Score Does not Lead to A Good Generative Model
  2. 2024Score Neural Operator: A Generative Model for Learning and Generalizing Across Multiple Probability Distributions
  3. 2023Closed-Form Diffusion Models
  4. 2023Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models
  5. 2022Score-Based Generative Models Detect Manifolds
  6. 2024Moderating the Generalization of Score-based Generative Model
  7. 2024An analysis of the noise schedule for score-based generative models
  8. 2022Wavelet Score-Based Generative Modeling