mlsanityData to 14 Oct 2026

Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models

arXiv 2405.14861 ↗First posted 23 May 2024cs.LG cs.AI math.STcited by 56

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

This paper investigates score-based diffusion models when the underlying target distribution is concentrated on or near low-dimensional manifolds within the higher-dimensional space in which they formally reside, a common characteristic of natural image distributions. Despite previous efforts to understand the data generation process of diffusion models, existing theoretical support remains highly suboptimal in the presence of low-dimensional structure, which we strengthen in this paper. For the popular Denoising Diffusion Probabilistic Model (DDPM), we find that the dependency of the error incurred within each denoising step on the ambient dimension d is in general unavoidable. We further identify a unique design of coefficients that yields a converges rate at the order of O(k2/√T) (up to log factors), where k is the intrinsic dimension of the target distribution and T is the number of steps. This represents the first theoretical demonstration that the DDPM sampler can adapt to unknown low-dimensional structures in the target distribution, highlighting the critical importance of coefficient design. All of this is achieved by a novel set of analysis tools that characterize the algorithmic dynamics in a more deterministic manner.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 18 of its 31 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

18 of 31 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  2. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  3. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  4. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  5. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  6. 2022Diffusion Models in Vision: A Survey10 Sep
  7. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  8. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  9. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  10. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  11. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 5

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  5. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 315

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 19; 2026: 23.

56 papers.

Sort:

By area:

  1. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 65
  2. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  3. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptionscited by 50
  4. 2024Linear Convergence of Diffusion Models Under the Manifold Hypothesiscited by 45
  5. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionalitycited by 44
  6. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 39
  7. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
  8. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  9. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  10. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  11. 2024Instance-dependent Convergence Theory for Diffusion Modelscited by 21
  12. 2025Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptivecited by 20
38 more
  1. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 18
  2. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  3. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  4. 2026High-accuracy sampling for diffusion models and log-concave distributionscited by 16
  5. 2025On the Interpolation Effect of Score Smoothing in Diffusion Modelscited by 15
  6. 2025A solvable model of learning generative diffusion: theory and insightscited by 14
  7. 2024Shallow diffusion networks provably learn hidden low-dimensional structurecited by 12
  8. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 10
  9. 2025Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Boundscited by 9
  10. 2025Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Modelscited by 9
  11. 2026Condition Errors Refinement in Autoregressive Image Generation with Diffusion Losscited by 9
  12. 2026Blind denoising diffusion models and the blessings of dimensionalitycited by 9
  13. 2025Provable Efficiency of Guidance in Diffusion Models for General Data Distributioncited by 8
  14. 2025Towards a unified framework for guided diffusion modelscited by 8
  15. 2025Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Modelscited by 7
  16. 2025Sublinear iterations can suffice even for DDPMscited by 7
  17. 2026High-accuracy and dimension-free sampling with diffusionscited by 7
  18. 2024Parallel Simulation for Log-concave Sampling and Score-based Diffusion Modelscited by 6
  19. 2024A Score-Based Density Formula, with Applications in Diffusion Generative Modelscited by 4
  20. 2026ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedulecited by 4
  21. 2026Provably adaptive sampling with uniform and remasking discrete diffusion modelscited by 4
  22. 2024BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inferencecited by 3
  23. 2025On the Contractivity of Stochastic Interpolation Flowcited by 3
  24. 2025Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Financecited by 3
  25. 2025Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networkscited by 3
  26. 2026Polynomial Convergence of Riemannian Diffusion Modelscited by 3
  27. 2026Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspectivecited by 3
  28. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 3
  29. 2026An exact information theory of generalization phase transitions in Bayesian diffusion modelscited by 3
  30. 2025Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Accesscited by 2
  31. 2025Optimal Stopping in Latent Diffusion Modelscited by 2
  32. 2026Wasserstein bounds for denoising diffusion probabilistic models via the F"ollmer processcited by 2
  33. 2026Denoising growth complexity: Data geometry and certified schedules for diffusion samplingcited by 2
  34. 2024Structural and Convergence Analysis of Discrete-Time Denoising Diffusion Probabilistic Modelscited by 1
  35. 2024On the Relation Between Linear Diffusion and Power Iterationcited by 1
  36. 2026On the Limits of Latent Reuse in Diffusion Modelscited by 1
  37. 2026Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structurescited by 1
  38. 2026Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choicescited by 1

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
  2. 2026Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Data
  3. 2026Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data
  4. 2023Interpreting and Improving Diffusion Models from an Optimization Perspective
  5. 2023Denoising Diffusion Samplers
  6. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
  7. 2025MAD: Manifold Attracted Diffusion
  8. 2025Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation