# Thermodynamic AI and the fluctuation frontier

arXiv: https://arxiv.org/abs/2302.06584 · HTML: https://mlsanity.com/paper/2302.06584/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 114 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Crystal Diffusion Variational Autoencoder for Periodic Material Generation](https://mlsanity.com/paper/2110.06197/) (2021)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Structure-based Drug Design with Equivariant Diffusion Models](https://mlsanity.com/paper/2210.13695/) (2022)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Noise-Induced Barren Plateaus in Variational Quantum Algorithms](https://mlsanity.com/paper/2007.14384/) (2020)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [Bayesian Neural Networks: An Introduction and Survey](https://mlsanity.com/paper/2006.12024/) (2020)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Theory for Equivariant Quantum Neural Networks](https://mlsanity.com/paper/2210.08566/) (2022)
- [Group-Invariant Quantum Machine Learning](https://mlsanity.com/paper/2205.02261/) (2022)
- [Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations](https://mlsanity.com/paper/2102.06559/) (2021)
- [Massively Parallel Probabilistic Computing with Sparse Ising Machines](https://mlsanity.com/paper/2110.02481/) (2021)
- [Stochastic noise can be helpful for variational quantum algorithms](https://mlsanity.com/paper/2210.06723/) (2022)
- [Can Error Mitigation Improve Trainability of Noisy Variational Quantum Algorithms?](https://mlsanity.com/paper/2109.01051/) (2021)
- [Thermodynamic Computing](https://mlsanity.com/paper/1911.01968/) (2019)
- [TherML: Thermodynamics of Machine Learning](https://mlsanity.com/paper/1807.04162/) (2018)
- [Thermodynamic Neural Network](https://mlsanity.com/paper/1906.01678/) (2019)
- [High-quality Thermal Gibbs Sampling with Quantum Annealing Hardware](https://mlsanity.com/paper/2109.01690/) (2021)
- [Thermodynamic Machine Learning through Maximum Work Production](https://mlsanity.com/paper/2006.15416/) (2020)

## Built on by (most cited first, 15 of 15)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Training Deep Boltzmann Networks with Sparse Ising Machines](https://mlsanity.com/paper/2303.10728/) (2023)
- [Thermodynamic Computing System for AI Applications](https://mlsanity.com/paper/2312.04836/) (2023)
- [Thermodynamic Computing via Autonomous Quantum Thermal Machines](https://mlsanity.com/paper/2308.15905/) (2023)
- [Thermodynamic Natural Gradient Descent](https://mlsanity.com/paper/2405.13817/) (2024)
- [Thermodynamic Bayesian Inference](https://mlsanity.com/paper/2410.01793/) (2024)
- [A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing](https://mlsanity.com/paper/2607.16183/) (2026)
- [Restrictions on Physical Stochastic Reservoir Computers](https://mlsanity.com/paper/2307.14474/) (2023)
- [Sub-universal variational circuits for combinatorial optimization problems](https://mlsanity.com/paper/2308.14981/) (2023)
- [Physical Foundation Models: Fixed hardware implementations of large-scale neural networks](https://mlsanity.com/paper/2604.27911/) (2026)
- [Agentic Information Theory: Ergodicity and Intrinsic Semantics of Information Processes](https://mlsanity.com/paper/2505.19275/) (2025)
- [Thermodynamics-Inspired Computing with Oscillatory Neural Networks for Inverse Matrix Computation](https://mlsanity.com/paper/2507.22544/) (2025)
- [Learning with springs and sticks](https://mlsanity.com/paper/2508.19015/) (2025)
- [CN101 - A Digital Thermodynamic Computer for Generative AI](https://mlsanity.com/paper/2608.00754/) (2026)

## Abstract (arXiv)

Many Artificial Intelligence (AI) algorithms are inspired by physics and employ stochastic fluctuations. We connect these physics-inspired AI algorithms by unifying them under a single mathematical framework that we call Thermodynamic AI. Seemingly disparate algorithmic classes can be described by this framework, for example, (1) Generative diffusion models, (2) Bayesian neural networks, (3) Monte Carlo sampling and (4) Simulated annealing. Such Thermodynamic AI algorithms are currently run on digital hardware, ultimately limiting their scalability and overall potential. Stochastic fluctuations naturally occur in physical thermodynamic systems, and such fluctuations can be viewed as a computational resource. Hence, we propose a novel computing paradigm, where software and hardware become inseparable. Our algorithmic unification allows us to identify a single full-stack paradigm, involving Thermodynamic AI hardware, that could accelerate such algorithms. We contrast Thermodynamic AI hardware with quantum computing where noise is a roadblock rather than a resource. Thermodynamic AI hardware can be viewed as a novel form of computing, since it uses a novel fundamental building block. We identify stochastic bits (s-bits) and stochastic modes (s-modes) as the respective building blocks for discrete and continuous Thermodynamic AI hardware. In addition to these stochastic units, Thermodynamic AI hardware employs a Maxwell's demon device that guides the system to produce non-trivial states. We provide a few simple physical architectures for building these devices and we develop a formalism for programming the hardware via gate sequences. We hope to stimulate discussion around this new computing paradigm. Beyond acceleration, we believe it will impact the design of both hardware and algorithms, while also deepening our understanding of the connection between physics and intelligence.