mlsanityData to 14 Oct 2026

Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models

arXiv 2309.16750 ↗First posted 28 Sep 2023cs.LG cs.AI math.DScited by 13

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks. Though the generative process is traditionally understood as an "iterative denoiser", there is no universally accepted language to describe it. We introduce a novel perspective to describe DMs using the mathematical language of memory retrieval from the field of energy-based Associative Memories (AMs), making efforts to keep our presentation approachable to newcomers to both of these fields. Unifying these two fields provides insight that DMs can be seen as a particular kind of AM where Lyapunov stability guarantees are bypassed by intelligently engineering the dynamics (i.e., the noise and step size schedules) of the denoising process. Finally, we present a growing body of evidence that records DMs exhibiting empirical behavior we would expect from AMs, and conclude by discussing research opportunities that are revealed by understanding DMs as a form of energy-based memory.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 44 of its 83 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

44 of 83 references.

Closest ancestors · 17

Same direction, newest first

  1. 2023Spontaneous Symmetry Breaking in Generative Diffusion Models31 May
  2. 2023Consistency Models2 Mar
  3. 2023On the Mathematics of Diffusion Models25 Jan
  4. 2022Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models21 Oct
  5. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  6. 2022Understanding Diffusion Models: A Unified Perspective25 Aug
  7. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise19 Aug
  8. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model16 Aug
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  10. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  11. 2021Score-Based Generative Classifiers1 Oct
  12. 2021Non Gaussian Denoising Diffusion Models14 Jun
5 more
  1. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  2. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  4. 2020Denoising Diffusion Probabilistic Models19 Jun
  5. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2018Neural Ordinary Differential Equationscited by 3,779
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  11. 2023Extracting Training Data from Diffusion Modelscited by 732
  12. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
15 more
  1. 2020Hopfield Networks is All You Needcited by 475
  2. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  3. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  4. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
  5. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  6. 2020Large Associative Memory Problem in Neurobiology and Machine Learningcited by 112
  7. 2021Relating transformers to models and neural representations of the hippocampal formationcited by 62
  8. 2022Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Modelscited by 55
  9. 2020Modern Hopfield Networks and Attention for Immune Repertoire Classificationcited by 53
  10. 2021Hierarchical Associative Memorycited by 34
  11. 2021Biological learning in key-value memory networkscited by 18
  12. 2021Can a Fruit Fly Learn Word Embeddings?cited by 17
  13. 2023Simplicial Hopfield networkscited by 16
  14. 2017Neural Distributed Autoassociative Memories: A Surveycited by 12
  15. 2022Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffoldcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 2; 2026: 6.

13 papers.

Sort:

By area:

  1. 2024On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)cited by 40
  2. 2025Memorization to Generalization: Emergence of Diffusion Models from Associative Memorycited by 36
  3. 2024Classifier-Free Guidance inside the Attraction Basin May Cause Memorizationcited by 21
  4. 2024Dense Associative Memory Through the Lens of Random Featurescited by 8
  5. 2025In-context denoising with one-layer transformers: connections between attention and associative memory retrievalcited by 7
  6. 2023Inverse distance weighting attentioncited by 4
  7. 2026Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balancecited by 3
  8. 2023A Waddington landscape for prototype learning in generalized Hopfield networkscited by 2
  9. 2026Deep Clustering with Associative Memoriescited by 2
  10. 2026Geodesic Flow Matching for Denoising High-Dimensional Structured Representationscited by 1
  11. 2026Self-Regulating Annealing in Heavy-Tailed Diffusion Modelscited by 1
  12. 2026Continual Learning in Modern Hopfield Networks with an Application to Diffusion Modelscited by 0
1 more
  1. 2026Explaining Machine Learning and Memorization with Statistical Mechanicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023In search of dispersed memories: Generative diffusion models are associative memory networks
  2. 2026Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
  3. 2026Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations
  4. 2026From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Models
  5. 2024Losing dimensions: Geometric memorization in generative diffusion
  6. 2026Cyclic Denoising Reveals Ultrastable Memories in Diffusion Models
  7. 2025Diffusion Models: A Mathematical Introduction
  8. 2026Early Signatures of Memorization in Diffusion Models via Basin Geometry and Cyclic Denoising