mlsanityData to 14 Oct 2026

The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability

arXiv 2310.17467 ↗First posted 26 Oct 2023stat.ML cs.LGcited by 39

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Generative diffusion models have achieved spectacular performance in many areas of machine learning and generative modeling. While the fundamental ideas behind these models come from non-equilibrium physics, variational inference and stochastic calculus, in this paper we show that many aspects of these models can be understood using the tools of equilibrium statistical mechanics. Using this reformulation, we show that generative diffusion models undergo second-order phase transitions corresponding to symmetry breaking phenomena. We show that these phase-transitions are always in a mean-field universality class, as they are the result of a self-consistency condition in the generative dynamics. We argue that the critical instability that arises from the phase transitions lies at the heart of their generative capabilities, which are characterized by a set of mean-field critical exponents. Finally, we show that the dynamic equation of the generative process can be interpreted as a stochastic adiabatic transformation that minimizes the free energy while keeping the system in thermal equilibrium.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 17 of its 37 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

17 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023In search of dispersed memories: Generative diffusion models are associative memory networks29 Sep
  2. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  3. 2023Spontaneous Symmetry Breaking in Generative Diffusion Models31 May
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  5. 2020Denoising Diffusion Implicit Models6 Oct
  6. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 11

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2022Video Diffusion Modelscited by 2,305
  3. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  5. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  6. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
  7. 2020Hopfield Networks is All You Needcited by 475
  8. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  9. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  10. 2022Sampling from the Sherrington-Kirkpatrick Gibbs measure via algorithmic stochastic localizationcited by 48
  11. 2023Sampling, Diffusions, and Stochastic Localizationcited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 10; 2026: 20.

39 papers.

Sort:

By area:

  1. 2024Dynamical Regimes of Diffusion Modelscited by 142
  2. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datacited by 93
  3. 2024Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusioncited by 32
  4. 2024Losing dimensions: Geometric memorization in generative diffusioncited by 29
  5. 2024Dynamic Negative Guidance of Diffusion Modelscited by 28
  6. 2024Probing the Latent Hierarchical Structure of Data via Diffusion Modelscited by 26
  7. 2025Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Modelscited by 25
  8. 2025On the Edge of Memorization in Diffusion Modelscited by 22
  9. 2025Entropic Time Schedulers for Generative Diffusion Modelscited by 16
  10. 2025Feedback Guidance of Diffusion Modelscited by 16
  11. 2025Provable Separations between Memorization and Generalization in Diffusion Modelscited by 14
  12. 2025The Information Dynamics of Generative Diffusioncited by 13
27 more
  1. 2026Theory of Speciation Transitions in Diffusion Models with General Class Structurecited by 10
  2. 2025Neural cellular automata: applications to biology and beyond classical AIcited by 9
  3. 2026The Entropic Signature of Class Speciation in Diffusion Modelscited by 9
  4. 2024Discrete generative diffusion models without stochastic differential equations: a tensor network approachcited by 6
  5. 2025Measuring Semantic Information Production in Generative Diffusion Modelscited by 6
  6. 2026How Out-of-Equilibrium Phase Transitions can Seed Pattern Formation in Trained Diffusion Modelscited by 6
  7. 2024Nonequilbrium physics of generative diffusion modelscited by 5
  8. 2026Dynamical Regimes of Multimodal Diffusion Modelscited by 4
  9. 2024Explosive neural networks via higher-order interactions in curved statistical manifoldscited by 3
  10. 2026Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balancecited by 3
  11. 2026Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Modelscited by 2
  12. 2026The Interplay of Data Structure and Imbalance in the Learning Dynamics of Diffusion Modelscited by 2
  13. 2026Remapping and navigation of an embedding space via error minimization: a fundamental organizational principle of cognition in natural and artificial systemscited by 1
  14. 2026Score Shocks: The Burgers Equation Structure of Diffusion Generative Modelscited by 1
  15. 2026Flowing with Confidencecited by 1
  16. 2026Self-Regulating Annealing in Heavy-Tailed Diffusion Modelscited by 1
  17. 2026Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometrycited by 1
  18. 2026A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifoldscited by 1
  19. 2026Adaptive Second-Order Solvers for Fast Stochastic Diffusion Samplingcited by 1
  20. 2025Exploring the Energy Landscape of RBMs: Reciprocal Space Insights into Bosons, Hierarchical Learning and Symmetry Breakingcited by 0
  21. 2025Enhancing diffusion models with Gaussianization preprocessingcited by 0
  22. 2026Continual Learning in Modern Hopfield Networks with an Application to Diffusion Modelscited by 0
  23. 2026Posterior Information Dynamics of Diffusion Models for Linear Inverse Problemscited by 0
  24. 2026The Loss Floor of Denoising Score Matching: Fisher Geometry from Schr"odinger Bridgescited by 0
  25. 2026Diffusion Models and Concept Formationcited by 0
  26. 2026Weighting Schedules Govern What and When Score-Based Generative Models Learn from Multimodal Datacited by 0
  27. 2026Your Discrete Graph Diffusion Model is Secretly Memorizingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Demystifying Variational Diffusion Models
  2. 2025A PDE Perspective on Generative Diffusion Models
  3. 2025Transition Models: Rethinking the Generative Learning Objective
  4. 2026Generative Modeling of Stochastic Dynamics for Long-Time Evolution
  5. 2025Thermodynamic Performance Limits for Score-Based Diffusion Models
  6. 2026Stochastic Thermodynamics and SDE-based Generative Models
  7. 2024Speed-accuracy relations for diffusion models: Wisdom from nonequilibrium thermodynamics and optimal transport
  8. 2026Some remarks on the connection between generative diffusion and optimal transport