# Memorization to Generalization: Emergence of Diffusion Models from Associative Memory

arXiv: https://arxiv.org/abs/2505.21777 · HTML: https://mlsanity.com/paper/2505.21777/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 21 of its 83 references that are in the map.
- First posted on arXiv 27 May 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 21 of 21)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Generalization in diffusion models arises from geometry-adaptive harmonic representations](https://mlsanity.com/paper/2310.02557/) (2023)
- [On Memorization in Diffusion Models](https://mlsanity.com/paper/2310.02664/) (2023)
- [An analytic theory of creativity in convolutional diffusion models](https://mlsanity.com/paper/2412.20292/) (2024)
- [Critical windows: non-asymptotic theory for feature emergence in diffusion models](https://mlsanity.com/paper/2403.01633/) (2024)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion](https://mlsanity.com/paper/2410.05898/) (2024)
- [Losing dimensions: Geometric memorization in generative diffusion](https://mlsanity.com/paper/2410.08727/) (2024)
- [Generalization through variance: how noise shapes inductive biases in diffusion models](https://mlsanity.com/paper/2504.12532/) (2025)
- [A solvable model of learning generative diffusion: theory and insights](https://mlsanity.com/paper/2501.03937/) (2025)
- [Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models](https://mlsanity.com/paper/2309.16750/) (2023)
- [Analysis of Diffusion Models for Manifold Data](https://mlsanity.com/paper/2502.04339/) (2025)
- [Random Features Hopfield Networks generalize retrieval to previously unseen examples](https://mlsanity.com/paper/2407.05658/) (2024)
- [Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry Breaking](https://mlsanity.com/paper/2406.12220/) (2024)

## Built on by (most cited first, 30 of 36)

- [Carr'e du champ flow matching: better quality-generalisation tradeoff in generative models](https://mlsanity.com/paper/2510.05930/) (2025)
- [The Information Dynamics of Generative Diffusion](https://mlsanity.com/paper/2508.19897/) (2025)
- [Generalization of Diffusion Models Arises with a Balanced Representation Space](https://mlsanity.com/paper/2512.20963/) (2025)
- [Generalization Dynamics of Linear Diffusion Models](https://mlsanity.com/paper/2505.24769/) (2025)
- [When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets](https://mlsanity.com/paper/2506.19031/) (2025)
- [Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasoners](https://mlsanity.com/paper/2606.29150/) (2026)
- [Local Coverage Governs Memorization in Diffusion Models](https://mlsanity.com/paper/2606.14390/) (2026)
- [Robust Noise Attenuation via Adaptive Pooling of Transformer Outputs](https://mlsanity.com/paper/2506.09215/) (2025)
- [Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balance](https://mlsanity.com/paper/2602.15914/) (2026)
- [An exact information theory of generalization phase transitions in Bayesian diffusion models](https://mlsanity.com/paper/2607.08041/) (2026)
- [Deep Clustering with Associative Memories](https://mlsanity.com/paper/2601.00963/) (2026)
- [The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks](https://mlsanity.com/paper/2603.25579/) (2026)
- [Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models](https://mlsanity.com/paper/2605.04830/) (2026)
- [Energy-Based Dynamical Models for Neurocomputation, Learning, and Optimization](https://mlsanity.com/paper/2604.05042/) (2026)
- [When Do Diffusion Models learn to Generate Multiple Objects?](https://mlsanity.com/paper/2605.00273/) (2026)
- [Memorisation, convergence and generalisation in generative models](https://mlsanity.com/paper/2605.21402/) (2026)
- [Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?](https://mlsanity.com/paper/2605.30642/) (2026)
- [Geodesic Flow Matching for Denoising High-Dimensional Structured Representations](https://mlsanity.com/paper/2606.00248/) (2026)
- [Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles](https://mlsanity.com/paper/2606.09718/) (2026)
- [Cyclic Denoising Reveals Ultrastable Memories in Diffusion Models](https://mlsanity.com/paper/2606.24000/) (2026)
- [What Neuroscience Can Teach AI About Learning in Continuously Changing Environments](https://mlsanity.com/paper/2507.02103/) (2025)
- [Coupled Inference in Diffusion Models for Semantic Decomposition](https://mlsanity.com/paper/2602.09983/) (2026)
- [On associative neural networks for sparse patterns with huge capacities](https://mlsanity.com/paper/2603.26217/) (2026)
- [Transformers Learn the Optimal DDPM Denoiser for Multi-Token GMMs](https://mlsanity.com/paper/2604.10074/) (2026)
- [Grokking of Diffusion Models: Case Study on Modular Addition](https://mlsanity.com/paper/2604.17673/) (2026)
- [On the Memorization of Consistency Distillation for Diffusion Models](https://mlsanity.com/paper/2604.23552/) (2026)
- [Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data](https://mlsanity.com/paper/2604.26841/) (2026)
- [Context-Gated Associative Retrieval: From Theory to Transformers](https://mlsanity.com/paper/2605.10970/) (2026)
- [The Silent Brush: Evaluating Artistic Style Leakage in AI Art Generation](https://mlsanity.com/paper/2605.17500/) (2026)
- [Adynamical systems view of training generativemodels and the memorization phenomenon](https://mlsanity.com/paper/2605.19483/) (2026)

## Abstract (arXiv)

Dense Associative Memories (DenseAMs) are generalizations of Hopfield networks, which have superior information storage capacity and can store training data points (memories) at local minima of the energy landscape. When the amount of training data exceeds the critical memory storage capacity of these models, new local minima, which are different from the training data, emerge. In Associative Memory these emergent local minima are called textitspurious; textitstates, which hinder memory retrieval. In this work, we examine diffusion models (DMs) through the DenseAM lens, viewing their generative process as an attempt of a memory retrieval. In the small data regimes, DMs create distinct attractors for each training sample, akin to DenseAMs below the critical memory storage. As the training data size increases, they transition from memorization to generalization. We identify a critical intermediate phase, predicted by DenseAM theory -- the spurious states. In generative modeling, these states are no longer negative artifacts but rather are the first signs of generative capabilities. We characterize the basins of attraction, energy landscape curvature, and computational properties of these previously overlooked states. Their existence is demonstrated across a wide range of architectures and datasets.