Thermodynamic cost of inference and learning in physical neural networks
Not on the map. Why?
Abstract (arXiv)
How much of the energy consumed by artificial neural networks is set by physics rather than by implementation? For irreversible digital hardware the reference is Landauer's principle, which charges kB Tln 2 per erased bit. We map a generic feedforward network onto a physical Hamiltonian in which each layer relation is an elastic compatibility constraint, and obtain two exact bounds. First, its equilibrium free energy is independent of the input and of every weight and bias, at all temperatures, so quasi-static inference requires no work whatsoever: no thermodynamic cost attaches to computation itself. Second, at finite speed the work exceeds the squared Wasserstein-2 distance between the initial and final thermal states, divided by the protocol duration. Relaxed to an entropic measure of distinguishability, this identifies the cost with the information separating successive inputs, about kB T per dimension of the widest layer at the fastest usable speed. Learning is fundamentally different: writing the parameters carries an irreducible cost of a few kB T each that survives the quasi-static limit. The thermodynamic price of a neural network is therefore set by its memory rather than its arithmetic. Simulations confirm both bounds: the inference work saturates the transport bound to within four percent, and accuracy collapses once the dissipated work falls below the thermal scale.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 57 references that are in the map. First posted on arXiv 13 Mar 2025.
Builds on
15 of 57 references.
References · 15
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
- 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 170
- 2019Efficient Learning for Deep Quantum Neural Networkscited by 161
- 2021Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemscited by 128
- 2020Photonics for artificial intelligence and neuromorphic computingcited by 112
- 2020Predictive Coding Approximates Backprop along Arbitrary Computation Graphscited by 90
- 2024Training of Physical Neural Networkscited by 49
- 2023Backpropagation-free Training of Deep Physical Neural Networkscited by 42
- 2021Self-learning Machines based on Hamiltonian Echo Backpropagationcited by 28
3 more
- 2023Thermodynamic Computing System for AI Applicationscited by 16
- 2024Nonlinear thermodynamic computing out of equilibriumcited by 4
- 2024Thermodynamic Bayesian Inferencecited by 2
Built on by
Not cited yet.