# On the Edge of Memorization in Diffusion Models

arXiv: https://arxiv.org/abs/2508.17689 · HTML: https://mlsanity.com/paper/2508.17689/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 59 of its 74 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 59)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models](https://mlsanity.com/paper/2306.04675/) (2023)
- [Generalization in diffusion models arises from geometry-adaptive harmonic representations](https://mlsanity.com/paper/2310.02557/) (2023)
- [A Self-Supervised Descriptor for Image Copy Detection](https://mlsanity.com/paper/2202.10261/) (2022)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [Dynamical Regimes of Diffusion Models](https://mlsanity.com/paper/2402.18491/) (2024)
- [On Memorization in Diffusion Models](https://mlsanity.com/paper/2310.02664/) (2023)
- [Ambient Diffusion: Learning Clean Distributions from Corrupted Data](https://mlsanity.com/paper/2305.19256/) (2023)
- [An analytic theory of creativity in convolutional diffusion models](https://mlsanity.com/paper/2412.20292/) (2024)
- [On Provable Copyright Protection for Generative Models](https://mlsanity.com/paper/2302.10870/) (2023)
- [Detecting, Explaining, and Mitigating Memorization in Diffusion Models](https://mlsanity.com/paper/2407.21720/) (2024)

## Built on by (most cited first, 22 of 22)

- [Provable Separations between Memorization and Generalization in Diffusion Models](https://mlsanity.com/paper/2511.03202/) (2025)
- [Making Reconstruction FID Predictive of Diffusion Generation FID](https://mlsanity.com/paper/2603.05630/) (2026)
- [Gradient Variance Reveals Failure Modes in Flow-Based Generative Models](https://mlsanity.com/paper/2510.18118/) (2025)
- [Dimension-free error estimate for diffusion model and optimal scheduling](https://mlsanity.com/paper/2512.01820/) (2025)
- [An exact information theory of generalization phase transitions in Bayesian diffusion models](https://mlsanity.com/paper/2607.08041/) (2026)
- [Transport Novelty Distance: A Distributional Metric for Evaluating Material Generative Models](https://mlsanity.com/paper/2512.09514/) (2025)
- [Diffusion Models Memorize in Training -- and Generalize in Inference](https://mlsanity.com/paper/2603.13419/) (2026)
- [Understanding diffusion models requires rethinking (again) generalization](https://mlsanity.com/paper/2605.06077/) (2026)
- [Memorization Control in Diffusion Models from Denoising-centric Perspective](https://mlsanity.com/paper/2601.21348/) (2026)
- [Reducing Memorisation in Generative Models via Riemannian Bayesian Inference](https://mlsanity.com/paper/2602.00199/) (2026)
- [Diffusion Processes on Implicit Manifolds](https://mlsanity.com/paper/2604.07213/) (2026)
- [A Quantitative Approximation Framework for Flow Distillation in Diffusion Models](https://mlsanity.com/paper/2606.03820/) (2026)
- [Efficient and Training-Free Single-Image Diffusion Models](https://mlsanity.com/paper/2606.04299/) (2026)
- [Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles](https://mlsanity.com/paper/2606.09718/) (2026)
- [Double Descent and Malign Overfitting in Diffusion Models](https://mlsanity.com/paper/2609.26392/) (2026)
- [Decomposing Private Image Generation via Coarse-to-Fine Wavelet Modeling](https://mlsanity.com/paper/2602.23262/) (2026)
- [Grokking of Diffusion Models: Case Study on Modular Addition](https://mlsanity.com/paper/2604.17673/) (2026)
- [On the Memorization of Consistency Distillation for Diffusion Models](https://mlsanity.com/paper/2604.23552/) (2026)
- [Adynamical systems view of training generativemodels and the memorization phenomenon](https://mlsanity.com/paper/2605.19483/) (2026)
- [A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Models](https://mlsanity.com/paper/2606.08554/) (2026)
- [Patched Flow Matching: Generative Wall-Pressure Reconstruction Beyond Training-Domain Scales from Sparse Sensors](https://mlsanity.com/paper/2606.22084/) (2026)
- [Background Gradients Shape Memorization in Flow Matching](https://mlsanity.com/paper/2609.33210/) (2026)

## Abstract (arXiv)

When do diffusion models reproduce their training data, and when are they able to generate samples beyond it? A practically relevant theoretical understanding of this interplay between memorization and generalization may significantly impact real-world deployments of diffusion models with respect to issues such as copyright infringement and data privacy. In this work, to disentangle the different factors that influence memorization and generalization in practical diffusion models, we introduce a scientific and mathematical "laboratory" for investigating these phenomena in diffusion models trained on fully synthetic or natural image-like structured data. Within this setting, we hypothesize that the memorization or generalization behavior of an underparameterized trained model is determined by the difference in training loss between an associated memorizing model and a generalizing model. To probe this hypothesis, we theoretically characterize a crossover point wherein the weighted training loss of a fully generalizing model becomes greater than that of an underparameterized memorizing model at a critical value of model (under)parameterization. We then demonstrate via carefully-designed experiments that the location of this crossover predicts a phase transition in diffusion models trained via gradient descent, validating our hypothesis. Ultimately, our theory enables us to analytically predict the model size at which memorization becomes predominant. Our work provides an analytically tractable and practically meaningful setting for future theoretical and empirical investigations. Code for our experiments is available at https://github.com/DruvPai/diffusion_mem_gen.