mlsanityData to 7 Oct 2026

Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality

arXiv 2410.18784 ↗First posted 24 Oct 2024cs.LG cs.NA eess.SPcited by 44

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The denoising diffusion probabilistic model (DDPM) has emerged as a mainstream generative model in generative AI. While sharp convergence guarantees have been established for the DDPM, the iteration complexity is, in general, proportional to the ambient data dimension, resulting in overly conservative theory that fails to explain its practical efficiency. This has motivated the recent work Li and Yan (2024a) to investigate how the DDPM can achieve sampling speed-ups through automatic exploitation of intrinsic low dimensionality of data. We strengthen this line of work by demonstrating, in some sense, optimal adaptivity to unknown low dimensionality. For a broad class of data distributions with intrinsic dimension k, we prove that the iteration complexity of the DDPM scales nearly linearly with k, which is optimal when using KL divergence to measure distributional discrepancy. Notably, our work is closely aligned with the independent concurrent work Potaptchik et al. (2024) -- posted two weeks prior to ours -- in establishing nearly linear-k convergence guarantees for the DDPM.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 36 of its 66 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

36 of 66 references.

Closest ancestors · 31

Same direction, newest first

  1. 2024Linear Convergence of Diffusion Models Under the Manifold Hypothesis11 Oct
  2. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models7 Oct
  3. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions27 Sep
  4. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions27 Sep
  5. 2024From optimal score matching to optimal sampling11 Sep
  6. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions4 Sep
  7. 2024Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing4 Sep
  8. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models5 Aug
  9. 2024Evaluating the design space of diffusion-based generative models18 Jun
  10. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models23 May
  11. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Models15 Apr
  12. 2024Accelerating Convergence of Score-Based Diffusion Models, Provably6 Mar
19 more
  1. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach21 Feb
  2. 2024Towards a mathematical theory for consistency training in diffusion models12 Feb
  3. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial12 Feb
  4. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances31 Jan
  5. 2023A Note on the Convergence of Denoising Diffusion Probabilistic Models10 Dec
  6. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  7. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  8. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  9. 2023The probability flow ODE is provably fast19 May
  10. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  11. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  12. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  13. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  14. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  15. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  16. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  17. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  18. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  19. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 5

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  5. 2021Intrinsic Dimension Estimation Using Wasserstein Distancescited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 18; 2026: 24.

44 papers.

Sort:

By area:

  1. 2024Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structurecited by 38
  2. 2024Provable Acceleration for Diffusion Models under Minimal Assumptionscited by 27
  3. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  4. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  5. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guaranteescited by 19
  6. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modelingcited by 18
  7. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  8. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  9. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  10. 2025On the Interpolation Effect of Score Smoothing in Diffusion Modelscited by 15
  11. 2026High-accuracy sampling for diffusion models and log-concave distributionscited by 13
  12. 2025Generalization of Diffusion Models Arises with a Balanced Representation Spacecited by 10
32 more
  1. 2025ANT: Adaptive Neural Temporal-Aware Text-to-Motion Modelcited by 9
  2. 2025Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Modelscited by 9
  3. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  4. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Modelscited by 8
  5. 2025Provable Efficiency of Guidance in Diffusion Models for General Data Distributioncited by 8
  6. 2025Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Boundscited by 8
  7. 2026Blind denoising diffusion models and the blessings of dimensionalitycited by 8
  8. 2025Sublinear iterations can suffice even for DDPMscited by 7
  9. 2025Towards a unified framework for guided diffusion modelscited by 7
  10. 2026High-accuracy and dimension-free sampling with diffusionscited by 7
  11. 2025Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysiscited by 6
  12. 2026Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexitycited by 6
  13. 2026Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Datacited by 4
  14. 2025Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Financecited by 3
  15. 2026Provably adaptive sampling with uniform and remasking discrete diffusion modelscited by 3
  16. 2025Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Accesscited by 2
  17. 2026Polynomial Convergence of Riemannian Diffusion Modelscited by 2
  18. 2026Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matchingcited by 2
  19. 2026Low-Dimensional Adaptation of Rectified Flow: A Diffusion and Stochastic Localization Perspectivecited by 2
  20. 2026Wasserstein bounds for denoising diffusion probabilistic models via the F"ollmer processcited by 2
  21. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2
  22. 2026Denoising growth complexity: Data geometry and certified schedules for diffusion samplingcited by 2
  23. 2025CINDES: Classification induced neural density estimator and simulatorcited by 1
  24. 2026On the Limits of Latent Reuse in Diffusion Modelscited by 1
  25. 2026Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choicescited by 1
  26. 2026Transformers Learn the Optimal DDPM Denoiser for Multi-Token GMMscited by 0
  27. 2026Causality-Encoded Diffusion Models for Interventional Sampling and Edge Inferencecited by 0
  28. 2026When Diffusion Model Can Ignore Dimension: An Entropy-Based Theorycited by 0
  29. 2026Connections between the F"ollmer process and the denoising diffusion probabilistic modelcited by 0
  30. 2026Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refinecited by 0
  31. 2026Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classificationcited by 0
  32. 2026A statistical theory for blind denoising: minimax estimation of the noise level from a single samplecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  2. 2023The Missing U for Efficient Diffusion Models
  3. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models
  4. 2021Improved Denoising Diffusion Probabilistic Models
  5. 2024Denoising Diffusion Probabilistic Models in Six Simple Steps
  6. 2025Diffusion Models: A Mathematical Introduction
  7. 2023Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis
  8. 2024An overview of diffusion models for generative artificial intelligence