# Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models

arXiv: https://arxiv.org/abs/2405.14861 · HTML: https://mlsanity.com/paper/2405.14861/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 23 May 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Convergence of denoising diffusion models under the manifold hypothesis](https://mlsanity.com/paper/2208.05314/) (2022)
- [Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization](https://mlsanity.com/paper/2308.03686/) (2023)
- [Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions](https://mlsanity.com/paper/2211.01916/) (2022)
- [Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data](https://mlsanity.com/paper/2302.07194/) (2023)
- [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023)

## Built on by (most cited first, 30 of 56)

- [A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models](https://mlsanity.com/paper/2408.02320/) (2024)
- [Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions](https://mlsanity.com/paper/2409.02426/) (2024)
- [O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions](https://mlsanity.com/paper/2409.18959/) (2024)
- [Linear Convergence of Diffusion Models Under the Manifold Hypothesis](https://mlsanity.com/paper/2410.09046/) (2024)
- [Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality](https://mlsanity.com/paper/2410.18784/) (2024)
- [Low-dimensional adaptation of diffusion models: Convergence in total variation](https://mlsanity.com/paper/2501.12982/) (2025)
- [Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models](https://mlsanity.com/paper/2410.04760/) (2024)
- [Provable Acceleration for Diffusion Models under Minimal Assumptions](https://mlsanity.com/paper/2410.23285/) (2024)
- [Minimax Optimality of the Probability Flow ODE for Diffusion Models](https://mlsanity.com/paper/2503.09583/) (2025)
- [Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures](https://mlsanity.com/paper/2504.05300/) (2025)
- [Instance-dependent Convergence Theory for Diffusion Models](https://mlsanity.com/paper/2410.13738/) (2024)
- [Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive](https://mlsanity.com/paper/2510.02305/) (2025)
- [Confidence-Based Decoding is Provably Efficient for Diffusion Language Models](https://mlsanity.com/paper/2603.22248/) (2026)
- [Faster Diffusion Models via Higher-Order Approximation](https://mlsanity.com/paper/2506.24042/) (2025)
- [Optimal Convergence Analysis of DDPM for General Distributions](https://mlsanity.com/paper/2510.27562/) (2025)
- [High-accuracy sampling for diffusion models and log-concave distributions](https://mlsanity.com/paper/2602.01338/) (2026)
- [On the Interpolation Effect of Score Smoothing in Diffusion Models](https://mlsanity.com/paper/2502.19499/) (2025)
- [A solvable model of learning generative diffusion: theory and insights](https://mlsanity.com/paper/2501.03937/) (2025)
- [Shallow diffusion networks provably learn hidden low-dimensional structure](https://mlsanity.com/paper/2410.11275/) (2024)
- [Adaptation to Intrinsic Dependence in Diffusion Language Models](https://mlsanity.com/paper/2602.20126/) (2026)
- [Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Bounds](https://mlsanity.com/paper/2506.09681/) (2025)
- [Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models](https://mlsanity.com/paper/2506.13061/) (2025)
- [Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss](https://mlsanity.com/paper/2602.07022/) (2026)
- [Blind denoising diffusion models and the blessings of dimensionality](https://mlsanity.com/paper/2602.09639/) (2026)
- [Provable Efficiency of Guidance in Diffusion Models for General Data Distribution](https://mlsanity.com/paper/2505.01382/) (2025)
- [Towards a unified framework for guided diffusion models](https://mlsanity.com/paper/2512.04985/) (2025)
- [Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2505.20789/) (2025)
- [Sublinear iterations can suffice even for DDPMs](https://mlsanity.com/paper/2511.04844/) (2025)
- [High-accuracy and dimension-free sampling with diffusions](https://mlsanity.com/paper/2601.10708/) (2026)
- [Parallel Simulation for Log-concave Sampling and Score-based Diffusion Models](https://mlsanity.com/paper/2412.07435/) (2024)

## Abstract (arXiv)

This paper investigates score-based diffusion models when the underlying target distribution is concentrated on or near low-dimensional manifolds within the higher-dimensional space in which they formally reside, a common characteristic of natural image distributions. Despite previous efforts to understand the data generation process of diffusion models, existing theoretical support remains highly suboptimal in the presence of low-dimensional structure, which we strengthen in this paper. For the popular Denoising Diffusion Probabilistic Model (DDPM), we find that the dependency of the error incurred within each denoising step on the ambient dimension d is in general unavoidable. We further identify a unique design of coefficients that yields a converges rate at the order of O(k2/√T) (up to log factors), where k is the intrinsic dimension of the target distribution and T is the number of steps. This represents the first theoretical demonstration that the DDPM sampler can adapt to unknown low-dimensional structures in the target distribution, highlighting the critical importance of coefficient design. All of this is achieved by a novel set of analysis tools that characterize the algorithmic dynamics in a more deterministic manner.