mlsanityData to 14 Oct 2026

Understanding diffusion models requires rethinking (again) generalization

arXiv 2605.06077 ↗First posted 7 May 2026cs.LGcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

This position paper argues that understanding generalization in diffusion models requires fundamentally new theoretical frameworks that go beyond both classical statistical learning theory and the benign overfitting paradigm developed for supervised learning. In diffusion models, unlike in supervised learning, memorization of training data and generalization to novel samples are incompatible: a model that has fully memorized its training set generates copies rather than novel data. Several theoretical explanations for why practical diffusion models nevertheless generalize have been proposed, based on capacity limitations, implicit regularization from optimization, or architectural inductive biases, but their interactions remain unclear. We argue that the field should pivot from explaining why the diffusion models do not memorize to investigating what the model actually learns during pre-memorization phase. To highlight our stance, we conduct empirical study of diffusion models trained on CIFAR-10, and we distill the findings into concrete open questions that we believe are key to improve understanding of generalization in diffusion models.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 52 of its 64 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

52 of 64 references.

Closest ancestors · 26

Same direction, newest first

  1. 2026Manifold Generalization Provably Proceeds Memorization in Diffusion Models24 Mar
  2. 2026A theory of learning data statistics in diffusion models, from easy to hard13 Mar
  3. 2025Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive2 Oct
  4. 2025When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis29 Sep
  5. 2025On the Edge of Memorization in Diffusion Models25 Aug
  6. 2025Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance5 Aug
  7. 2025Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis4 Jul
  8. 2025Generalization Dynamics of Linear Diffusion Models30 May
  9. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training23 May
  10. 2025Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models22 May
  11. 2025Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces5 May
  12. 2025Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization5 Feb
14 more
  1. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions27 Sep
  2. 2024Dynamical Regimes of Diffusion Models28 Feb
  3. 2024An analysis of the noise schedule for score-based generative models7 Feb
  4. 2024A Good Score Does not Lead to A Good Generative Model10 Jan
  5. 2023On the Limitation of Diffusion Models for Synthesizing Training Datasets22 Nov
  6. 2023On Memorization in Diffusion Models4 Oct
  7. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  8. 2023KL Convergence Guarantees for Score diffusion models under minimal data assumptions23 Aug
  9. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  10. 2023Understanding and Mitigating Copying in Diffusion Models31 May
  11. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  12. 2022Classifier-Free Diffusion Guidance26 Jul
  13. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  14. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 26

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  4. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  5. 2017A Closer Look at Memorization in Deep Networkscited by 942
  6. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  7. 2023Extracting Training Data from Diffusion Modelscited by 732
  8. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  9. 2017Exploring Generalization in Deep Learningcited by 600
  10. 2017The Perception-Distortion Tradeoffcited by 587
  11. 2019Benign Overfitting in Linear Regressioncited by 549
  12. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
14 more
  1. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
  2. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  3. 2017Implicit Regularization in Matrix Factorizationcited by 322
  4. 2015A note on the evaluation of generative modelscited by 292
  5. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 268
  6. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  7. 2022A Self-Supervised Descriptor for Image Copy Detectioncited by 176
  8. 2024An analytic theory of creativity in convolutional diffusion modelscited by 104
  9. 2023On Provable Copyright Protection for Generative Modelscited by 98
  10. 2024Understanding Optimization in Deep Learning with Central Flowscited by 64
  11. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  12. 2023On Copyright Risks of Text-to-Image Diffusion Modelscited by 20
  13. 2024Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizescited by 12
  14. 2023Exact Mean Square Linear Stability Analysis for SGDcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stabilitycited by 3
  2. 2026Double Descent and Malign Overfitting in Diffusion Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023The Emergence of Reproducibility and Generalizability in Diffusion Models
  2. 2026Diffusion Models Memorize in Training -- and Generalize in Inference
  3. 2023On the Generalization Properties of Diffusion Models
  4. 2025Generalization of Diffusion Models Arises with a Balanced Representation Space
  5. 2025How Diffusion Models Memorize
  6. 2025Provable Separations between Memorization and Generalization in Diffusion Models
  7. 2025Generalization through variance: how noise shapes inductive biases in diffusion models
  8. 2026Local Coverage Governs Memorization in Diffusion Models