# Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models

arXiv: https://arxiv.org/abs/2605.09031 · HTML: https://mlsanity.com/paper/2605.09031/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 74 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 12 of 12)

- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Scaling Laws and Interpretability of Learning from Repeated Data](https://mlsanity.com/paper/2205.10487/) (2022)
- [How much do language models memorize?](https://mlsanity.com/paper/2505.24832/) (2025)
- [Bayesian GAN](https://mlsanity.com/paper/1705.09558/) (2017)
- [There Will Be a Scientific Theory of Deep Learning](https://mlsanity.com/paper/2604.21691/) (2026)
- [Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networks](https://mlsanity.com/paper/2502.21269/) (2025)
- [Double Descent and Other Interpolation Phenomena in GANs](https://mlsanity.com/paper/2106.04003/) (2021)
- [Unveiling Multiple Descents in Unsupervised Autoencoders](https://mlsanity.com/paper/2406.11703/) (2024)
- [Understanding Generalization in Diffusion Distillation via Probability Flow Distance](https://mlsanity.com/paper/2505.20123/) (2025)
- [Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions](https://mlsanity.com/paper/2505.18046/) (2025)
- [Double Descent as a Lens for Sample Efficiency in Autoregressive vs. Discrete Diffusion Models](https://mlsanity.com/paper/2509.24974/) (2025)
- [Understanding temperature tuning in energy-based models](https://mlsanity.com/paper/2512.09152/) (2025)

## Abstract (arXiv)

Energy-based models (EBMs) are flexible generative architectures inspired by statistical physics, but their learning and generative properties remain poorly understood. Here, we analyze a solvable EBM in the high-dimensional limit: the spherical Boltzmann machine (SBM). Combining tools from random matrix theory and dynamical mean-field theory, we: solve exact equations describing the training dynamics of the SBM; compute the Bayesian evidence, which acts as a partition function in parameter space and encodes global properties of the trained model; and uncover cascades of phase transitions that occur both during training and as a function of hyperparameters, related to successive alignment and condensation of the top modes of the coupling matrix to the data. We connect these transitions to sampling-time generative phenomena in a teacher-student scenario, including: sampling temperature tuning, double descent as a function of regularization strength, tempered posterior effects, and out-of-equilibrium effects during training that induce biases in the trained model. We provide numerical evidence demonstrating that all these phenomena appear in standard generative architectures, beyond the SBM.