mlsanityData to 7 Oct 2026

Shallow diffusion networks provably learn hidden low-dimensional structure

arXiv 2410.11275 ↗First posted 15 Oct 2024cs.LG stat.MLcited by 12

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion-based generative models provide a powerful framework for learning to sample from a complex target distribution. The remarkable empirical success of these models applied to high-dimensional signals, including images and video, stands in stark contrast to classical results highlighting the curse of dimensionality for distribution recovery. In this work, we take a step towards understanding this gap through a careful analysis of learning diffusion models over the Barron space of single layer neural networks. In particular, we show that these shallow models provably adapt to simple forms of low dimensional structure, thereby avoiding the curse of dimensionality. We combine our results with recent analyses of sampling with diffusion models to provide an end-to-end sample complexity bound for learning to sample from structured distributions. Importantly, our results do not require specialized architectures tailored to particular latent structures, and instead rely on the low-index structure of the Barron space to adapt to the underlying distribution.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 46 of its 65 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

46 of 65 references.

Closest ancestors · 20

Same direction, newest first

  1. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions27 Sep
  2. 2024From optimal score matching to optimal sampling11 Sep
  3. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models5 Aug
  4. 2024Evaluating the design space of diffusion-based generative models18 Jun
  5. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models23 May
  6. 2024Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions23 Feb
  7. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances31 Jan
  8. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization28 Jan
  9. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  10. 2023The probability flow ODE is provably fast19 May
  11. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  12. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
8 more
  1. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  2. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  3. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  4. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  5. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  6. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov

Other references · 26

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  4. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  5. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  6. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  7. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  8. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  9. 2021Bilinear Classes: A Structural Framework for Provable Generalization in RLcited by 176
  10. 2016A vector-contraction inequality for Rademacher complexitiescited by 141
  11. 2022The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networkscited by 141
  12. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
14 more
  1. 2019A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Normcited by 105
  2. 2022Learning Single-Index Models with Shallow Neural Networkscited by 92
  3. 2023Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamicscited by 72
  4. 2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 69
  5. 2023How Two-Layer Neural Networks Learn, One (Giant) Step at a Timecited by 67
  6. 2022Statistical Efficiency of Score Matching: The View from Isoperimetrycited by 63
  7. 2024Optimal score estimation via empirical Bayes smoothingcited by 60
  8. 2014Learning without Concentrationcited by 50
  9. 2024Neural network learns low-dimensional polynomials with SGD near the information-theoretic limitcited by 45
  10. 2019Mean Field Analysis of Deep Neural Networkscited by 42
  11. 2019The Barron Space and the Flow-induced Function Spaces for Neural Network Modelscited by 39
  12. 2023SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problemcited by 36
  13. 2021On Energy-Based Models with Overparametrized Shallow Neural Networkscited by 8
  14. 2024How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Controlcited by 35
  2. 2025On the Interpolation Effect of Score Smoothing in Diffusion Modelscited by 15
  3. 2025A solvable model of learning generative diffusion: theory and insightscited by 14
  4. 2025Sublinear iterations can suffice even for DDPMscited by 7
  5. 2025Kernel-Smoothed Scores for Denoising Diffusion: A Bias-Variance Studycited by 3
  6. 2026An exact information theory of generalization phase transitions in Bayesian diffusion modelscited by 3
  7. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2
  8. 2025Dimension-free Score Matching and Time Bootstrapping for Diffusion Modelscited by 1
  9. 2026Exploring Time Conditioning in Diffusion Generative Models from Disjoint Noisy Data Manifoldscited by 1
  10. 2026Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choicescited by 1
  11. 2026Transformers Learn the Optimal DDPM Denoiser for Multi-Token GMMscited by 0
  12. 2026Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refinecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions
  2. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  3. 2025Diffusion Models: A Mathematical Introduction
  4. 2022Understanding Diffusion Models: A Unified Perspective
  5. 2023Neural Diffusion Models
  6. 2024Analyzing Neural Network-Based Generative Diffusion Models through Convex Optimization
  7. 2025Nonparametric estimation of a factorizable density using diffusion models
  8. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models