# Neural Thermodynamic Laws for Large Language Model Training

arXiv: https://arxiv.org/abs/2505.10559 · HTML: https://mlsanity.com/paper/2505.10559/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2025.

## Built on by (most cited first, 10 of 10)

- [Universal One-third Time Scaling in Learning Peaked Distributions](https://mlsanity.com/paper/2602.03685/) (2026)
- [Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Model](https://mlsanity.com/paper/2602.04774/) (2026)
- [Detailed balance in large language model-driven agents](https://mlsanity.com/paper/2512.10047/) (2025)
- [From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretability](https://mlsanity.com/paper/2603.29805/) (2026)
- [Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agents](https://mlsanity.com/paper/2604.00555/) (2026)
- [Mpemba Effect in Large-Language Model Training Dynamics: A Minimal Analysis of the Valley-River model](https://mlsanity.com/paper/2507.04206/) (2025)
- [QLENS: Towards A Quantum Perspective of Language Transformers](https://mlsanity.com/paper/2510.11963/) (2025)
- [Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?](https://mlsanity.com/paper/2511.07308/) (2025)
- [The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow](https://mlsanity.com/paper/2609.01034/) (2026)
- [Optimizer-dependent training dynamics converge to the same one-third optimal data scaling](https://mlsanity.com/paper/2609.37745/) (2026)

## Abstract (arXiv)

Beyond neural scaling laws, little is known about the laws underlying large language models (LLMs). We introduce Neural Thermodynamic Laws (NTL) -- a new framework that offers fresh insights into LLM training dynamics. On the theoretical side, we demonstrate that key thermodynamic quantities (e.g., temperature, entropy, heat capacity, thermal conduction) and classical thermodynamic principles (e.g., the three laws of thermodynamics and the equipartition theorem) naturally emerge under river-valley loss landscape assumptions. On the practical side, this scientific perspective yields intuitive guidelines for designing learning rate schedules.