mlsanityData to 13 Oct 2026

Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curves

arXiv 2502.00336 ↗First posted 1 Feb 2025cs.LG stat.MLcited by 20

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We theoretically investigate the phenomena of generalization and memorization in diffusion models. Empirical studies suggest that these phenomena are influenced by model complexity and the size of the training dataset. In our experiments, we further observe that the number of noise samples per data sample (m) used during Denoising Score Matching (DSM) plays a significant and non-trivial role. We capture these behaviors and shed insights into their mechanisms by deriving asymptotically precise expressions for test and train errors of DSM under a simple theoretical setting. The score function is parameterized by random features neural networks, with the target distribution being d-dimensional Gaussian. We operate in a regime where the dimension d, number of data samples n, and number of features p tend to infinity while keeping the ratios psin=fracnd and psip=fracpd fixed. By characterizing the test and train errors, we identify regimes of generalization and memorization as a function of psin,psip, and m. Our theoretical findings are consistent with the empirical observations.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 45 of its 66 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

45 of 66 references.

Closest ancestors · 27

Same direction, newest first

  1. 2024Losing dimensions: Geometric memorization in generative diffusion11 Oct
  2. 2024Evaluating the design space of diffusion-based generative models18 Jun
  3. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion models3 Mar
  4. 2024Dynamical Regimes of Diffusion Models28 Feb
  5. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data26 Feb
  6. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization28 Jan
  7. 2023On the Generalization Properties of Diffusion Models3 Nov
  8. 2023Generalization Bound for Diffusion Models using Random Features6 Oct
  9. 2023On Memorization in Diffusion Models4 Oct
  10. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  11. 2023In search of dispersed memories: Generative diffusion models are associative memory networks29 Sep
  12. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
15 more
  1. 2023Learning Mixtures of Gaussians Using the DDPM Objective3 Jul
  2. 2023Understanding and Mitigating Copying in Diffusion Models31 May
  3. 2023Spontaneous Symmetry Breaking in Generative Diffusion Models31 May
  4. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  5. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  6. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  7. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  8. 2022Classifier-Free Diffusion Guidance26 Jul
  9. 2022Score-Based Generative Models Detect Manifolds2 Jun
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  11. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  13. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  14. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  15. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,656
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,077
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
  6. 2020Extracting Training Data from Large Language Modelscited by 2,181
  7. 2023Extracting Training Data from Diffusion Modelscited by 729
  8. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 421
  9. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 411
  10. 2020Generalisation error in learning with random features and the hidden manifold modelcited by 126
  11. 2020The Gaussian equivalence of generative models for learning with shallow neural networkscited by 113
  12. 2024An analytic theory of creativity in convolutional diffusion modelscited by 103
6 more
  1. 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
  2. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  3. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  4. 2024Asymptotics of Random Feature Regression Beyond the Linear Scaling Regimecited by 22
  5. 2021Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature modelcited by 17
  6. 2015Applications of Realizations (aka Linearizations) to Free Probabilitycited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 6; 2026: 14.

20 papers.

Sort:

By area:

  1. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Trainingcited by 79
  2. 2025Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Modelscited by 25
  3. 2025On the Edge of Memorization in Diffusion Modelscited by 22
  4. 2025Provable Separations between Memorization and Generalization in Diffusion Modelscited by 14
  5. 2025Generalization of Diffusion Models Arises with a Balanced Representation Spacecited by 10
  6. 2025Generalization Dynamics of Linear Diffusion Modelscited by 9
  7. 2026A Random Matrix Theory Perspective on the Consistency of Diffusion Modelscited by 5
  8. 2026Diffusion Model's Generalization Can Be Characterized by Inductive Biases toward a Data-Dependent Ridge Manifoldcited by 5
  9. 2026Local Coverage Governs Memorization in Diffusion Modelscited by 5
  10. 2026Biased Generalization in Diffusion Modelscited by 3
  11. 2026Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Datacited by 3
  12. 2026Benign Overfitting Does Not Occur in Diffusion Modelscited by 3
8 more
  1. 2026Diffusion Models Memorize in Training -- and Generalize in Inferencecited by 2
  2. 2026The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Modelscited by 2
  3. 2026Memorisation, convergence and generalisation in generative modelscited by 1
  4. 2026Double Descent and Malign Overfitting in Diffusion Modelscited by 1
  5. 2026On the Memorization of Consistency Distillation for Diffusion Modelscited by 0
  6. 2026Reducing Diffusion Model Memorization with Higher Order Langevin Dynamicscited by 0
  7. 2026Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Datacited by 0
  8. 2026First Learn, Then Memorize: The Spectral Bias of Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization
  2. 2025Generalization through variance: how noise shapes inductive biases in diffusion models
  3. 2025Heteroscedasticity of Denoising Score Matching with Generalised Smooth Noise
  4. 2022From Denoising Diffusions to Denoising Markov Models
  5. 2024Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure
  6. 2025Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis
  7. 2026Memorization and Malign Generalization in Conditional Diffusion Models with Random Features
  8. 2024What's the score? Automated Denoising Score Matching for Nonlinear Diffusions