mlsanityData to 7 Oct 2026

Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data

arXiv 2302.07194 ↗First posted 14 Feb 2023cs.LG stat.MLcited by 171

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models achieve state-of-the-art performance in various generation tasks. However, their theoretical foundations fall far behind. This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. Our result provides sample complexity bounds for distribution estimation using diffusion models. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. Furthermore, the generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution. The convergence rate depends on the subspace dimension, indicating that diffusion models can circumvent the curse of data ambient dimensionality.

Key facts. Cited by 171 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 34 of its 53 references that are in the map. First posted on arXiv 14 Feb 2023.

Builds on

34 of 53 references.

Closest ancestors · 14

Same direction, newest first

  1. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  2. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  3. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models31 Aug
  4. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  5. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  6. 2022Score-Based Generative Models Detect Manifolds2 Jun
  7. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Models27 May
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2021Score-based Generative Modeling in Latent Space10 Jun
  10. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation10 Jun
  11. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
2 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun
  2. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2015Variational Inference with Normalizing Flowscited by 2,161
  7. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  8. 2016Error bounds for approximations with deep ReLU networkscited by 567
  9. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  10. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  11. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  12. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
8 more
  1. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  2. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  3. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  4. 2020Training robust neural networks using Lipschitz boundscited by 98
  5. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  6. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  7. 2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
  8. 2019On Generalization Bounds of a Family of Recurrent Neural Networkscited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 57; 2025: 55; 2026: 43.

171 papers.

Sort:

By area:

  1. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  2. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localizationcited by 230
  3. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Modelscited by 156
  4. 2023Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problemscited by 129
  5. 2024An analytic theory of creativity in convolutional diffusion modelscited by 103
  6. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datacited by 88
  7. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Modelscited by 84
  8. 2023DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuningcited by 82
  9. 2025ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceivercited by 80
  10. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  11. 2024Accelerating Diffusion Sampling with Optimized Time Stepscited by 67
  12. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 64
38 more
  1. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
  2. 2024Optimal score estimation via empirical Bayes smoothingcited by 60
  3. 2024Gradient Guidance for Diffusion Models: An Optimization Perspectivecited by 59
  4. 2024Rectified Diffusion: Straightness Is Not Your Need in Rectified Flowcited by 59
  5. 2024Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theorycited by 58
  6. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distancescited by 56
  7. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Modelscited by 56
  8. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Modelscited by 56
  9. 2024Reconstructive Visual Instruction Tuningcited by 54
  10. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  11. 2023Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvementcited by 49
  12. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptionscited by 48
  13. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  14. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Modelscited by 46
  15. 2024What does guidance do? A fine-grained analysis in a simple settingcited by 45
  16. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionalitycited by 44
  17. 2024Learning Mixtures of Gaussians Using Diffusion Modelscited by 41
  18. 2023Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Modelscited by 39
  19. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalizationcited by 39
  20. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  21. 2024Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structurecited by 38
  22. 2024A Good Score Does not Lead to A Good Generative Modelcited by 36
  23. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
  24. 2024Contractive Diffusion Probabilistic Modelscited by 29
  25. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  26. 2024Evaluating the design space of diffusion-based generative modelscited by 25
  27. 2023On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimatescited by 24
  28. 2024Unifying Generation and Prediction on Graphs with Latent Graph Diffusioncited by 24
  29. 2024Diffusion Model for Data-Driven Black-Box Optimizationcited by 24
  30. 2024Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspectivecited by 24
  31. 2025How Compositional Generalization and Creativity Improve as Diffusion Models are Trainedcited by 24
  32. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  33. 2025Generalization through variance: how noise shapes inductive biases in diffusion modelscited by 23
  34. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  35. 2023Distribution learning via neural differential equations: a nonparametric statistical perspectivecited by 22
  36. 2023The Emergence of Reproducibility and Generalizability in Diffusion Modelscited by 21
  37. 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
  38. 2025Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curvescited by 20

50 most cited of 171 All 171 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data
  2. 2023Diffusion Models are Minimax Optimal Distribution Estimators
  3. 2022Subspace Diffusion Generative Models
  4. 2024Analyzing Neural Network-Based Generative Diffusion Models through Convex Optimization
  5. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  6. 2024The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications
  7. 2024Nearest Neighbour Score Estimators for Diffusion Generative Models
  8. 2025Dimension-free Score Matching and Time Bootstrapping for Diffusion Models