mlsanityData to 7 Oct 2026

Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Data

arXiv 2603.22962 ↗First posted 24 Mar 2026cs.LG stat.MLcited by 3

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We study the theoretical behavior of denoising score matching--the learning task associated to diffusion models--when the data distribution is supported on a low-dimensional manifold and the score is parameterized using a random feature neural network. We derive asymptotically exact expressions for the test, train, and score errors in the high-dimensional limit. Our analysis reveals that, for linear manifolds the sample complexity required to learn the score function scales linearly with the intrinsic dimension of the manifold, rather than with the ambient dimension. Perhaps surprisingly, the benefits of low-dimensional structure starts to diminish once we have a non-linear manifold. These results indicate that diffusion models can benefit from structured data; however, the dependence on the specific type of structure is subtle and intricate.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 37 of its 55 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

37 of 55 references.

Closest ancestors · 27

Same direction, newest first

  1. 2025When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets23 Jun
  2. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training23 May
  3. 2025Analysis of Diffusion Models for Manifold Data1 Feb
  4. 2025Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curves1 Feb
  5. 2024Losing dimensions: Geometric memorization in generative diffusion11 Oct
  6. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions27 Sep
  7. 2024Evaluating the design space of diffusion-based generative models18 Jun
  8. 2024Dynamical Regimes of Diffusion Models28 Feb
  9. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization28 Jan
  10. 2023On the Generalization Properties of Diffusion Models3 Nov
  11. 2023Generalization Bound for Diffusion Models using Random Features6 Oct
  12. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
15 more
  1. 2023In search of dispersed memories: Generative diffusion models are associative memory networks29 Sep
  2. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  3. 2023Learning Mixtures of Gaussians Using the DDPM Objective3 Jul
  4. 2023Spontaneous Symmetry Breaking in Generative Diffusion Models31 May
  5. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  6. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  7. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  8. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  9. 2022Classifier-Free Diffusion Guidance26 Jul
  10. 2022Score-Based Generative Models Detect Manifolds2 Jun
  11. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  13. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  14. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  15. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 10

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  5. 2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 115
  6. 2020The Gaussian equivalence of generative models for learning with shallow neural networkscited by 113
  7. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  8. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  9. 2021Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature modelcited by 17
  10. 2022Gradient flow in the gaussian covariate model: exact solution of learning curves and multiple descent structurescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Benign Overfitting Does Not Occur in Diffusion Modelscited by 3
  2. 2026Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refinecited by 0
  3. 2026Memorization and Malign Generalization in Conditional Diffusion Models with Random Featurescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity
  2. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models
  3. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions
  4. 2025MAD: Manifold Attracted Diffusion
  5. 2026Manifold Generalization Provably Proceeds Memorization in Diffusion Models
  6. 2026Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data
  7. 2025Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive
  8. 2025On the Interpolation Effect of Score Smoothing in Diffusion Models