Thermodynamic Performance Limits for Score-Based Diffusion Models
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
We establish a fundamental connection between score-based diffusion models and non-equilibrium thermodynamics by deriving performance limits based on entropy rates. Our main theoretical contribution is a lower bound on the negative log-likelihood of the data that relates model performance to entropy rates of diffusion processes. We numerically validate this bound on a synthetic dataset and investigate its tightness. By building a bridge to entropy rates - system, intrinsic, and exchange entropy - we provide new insights into the thermodynamic operation of these models, drawing parallels to Maxwell's demon and implications for thermodynamic computing hardware. Our framework connects generative modeling performance to fundamental physical principles through stochastic thermodynamics.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 9 of its 15 references that are in the map. First posted on arXiv 7 Oct 2025.
Builds on
9 of 15 references.
Closest ancestors · 4
Same direction, newest first
- 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance13 Dec
- 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
- 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
- 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
Other references · 5
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
- 2022Generative Modelling With Inverse Heat Dissipationcited by 167
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Speed-accuracy relations for diffusion models: Wisdom from nonequilibrium thermodynamics and optimal transport
- 2025A PDE Perspective on Generative Diffusion Models
- 2024Nonequilbrium physics of generative diffusion models
- 2026Stochastic Thermodynamics and SDE-based Generative Models
- 2026Symmetric Equilibrium Propagation for Thermodynamic Diffusion Training
- 2026Modeling Score Approximation Errors in Diffusion Models via Forward SPDEs
- 2026Improved Sampling Schedules for Discrete Diffusion Models
- 2023The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability