# Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regime

arXiv: https://arxiv.org/abs/2608.23938 · HTML: https://mlsanity.com/paper/2608.23938/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 13 of its 111 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 13 of 13)

- [An analytic theory of creativity in convolutional diffusion models](https://mlsanity.com/paper/2412.20292/) (2024)
- [Memorization and Regularization in Generative Diffusion Models](https://mlsanity.com/paper/2501.15785/) (2025)
- [Losing dimensions: Geometric memorization in generative diffusion](https://mlsanity.com/paper/2410.08727/) (2024)
- [A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimator](https://mlsanity.com/paper/2403.08938/) (2024)
- [From optimal score matching to optimal sampling](https://mlsanity.com/paper/2409.07032/) (2024)
- [Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models](https://mlsanity.com/paper/2505.16959/) (2025)
- [The Hidden Linear Structure in Score-Based Models and its Application](https://mlsanity.com/paper/2311.10892/) (2023)
- [Quantitative deterministic equivalent of sample covariance matrices with a general dependence structure](https://mlsanity.com/paper/2211.13044/) (2022)
- [A Random Matrix Theory Perspective on the Consistency of Diffusion Models](https://mlsanity.com/paper/2602.02908/) (2026)
- [Local Coverage Governs Memorization in Diffusion Models](https://mlsanity.com/paper/2606.14390/) (2026)
- [Anisotropic local law for non-separable sample covariance matrices](https://mlsanity.com/paper/2602.17960/) (2026)
- [Benign Overfitting Does Not Occur in Diffusion Models](https://mlsanity.com/paper/2607.02671/) (2026)
- [Diffusion Models Memorize in Training -- and Generalize in Inference](https://mlsanity.com/paper/2603.13419/) (2026)

## Built on by (most cited first, 3 of 3)

- [Double Descent and Malign Overfitting in Diffusion Models](https://mlsanity.com/paper/2609.26392/) (2026)
- [First Learn, Then Memorize: The Spectral Bias of Diffusion Models](https://mlsanity.com/paper/2609.35377/) (2026)
- [Memorization and Malign Generalization in Conditional Diffusion Models with Random Features](https://mlsanity.com/paper/2610.11288/) (2026)

## Abstract (arXiv)

Modern score-based generative models have achieved remarkable empirical success in high-dimensional tasks such as image, audio, and video synthesis. These models reduce distribution learning to a sequence of regression problems that, if solved exactly on finite data, would ultimately reproduce the training samples. Their ability to generalize must therefore arise from the implicit or explicit regularization during training. In this work, we develop a generative counterpart to the theory of benign overfitting and algorithmic regularization for overparameterized neural networks in the supervised lazy-training regime. We study denoising score matching in a vector-valued reproducing kernel Hilbert space with an inner-product kernel. In the proportional high-dimensional regime nasymp d, we derive exact risk trajectories under gradient flow training. These trajectories exhibit three phases governed by qualitatively distinct estimators: a spectral estimator that generalizes, a pure-noise score with localized peaks that interpolate the training objective, and an empirical Bayes estimator that memorizes the data. We then analyze how these estimators combine along the reverse-time SDE and characterize the distribution of the resulting samples. The analysis reveals familiar mechanisms from supervised learning, including kernel linearization and self-induced regularization from the nonlinear part of the kernel, but also reveals a distinct phenomenology specific to generative modeling.