# Toward a Physics of Deep Learning and Brains

arXiv: https://arxiv.org/abs/2509.22649 · HTML: https://mlsanity.com/paper/2509.22649/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mean Field Residual Networks: On the Edge of Chaos](https://mlsanity.com/paper/1712.08969/) (2017)
- [Determination of the edge of criticality in echo state networks through Fisher information maximization](https://mlsanity.com/paper/1603.03685/) (2016)
- [Edge of chaos as a guiding principle for modern neural network training](https://mlsanity.com/paper/2107.09437/) (2021)
- [Optimal Machine Intelligence at the Edge of Chaos](https://mlsanity.com/paper/1909.05176/) (2019)
- [Feature Learning and Generalization in Deep Networks with Orthogonal Weights](https://mlsanity.com/paper/2310.07765/) (2023)
- [Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networks](https://mlsanity.com/paper/2307.02284/) (2023)
- [Quasi-Critical Brain Dynamics on a Non-Equilibrium Widom Line](https://mlsanity.com/paper/1405.6232/) (2014)
- [Critical dynamics governs deep learning](https://mlsanity.com/paper/2507.08527/) (2025)
- [Quasicriticality explains variability of human neural dynamics across life span](https://mlsanity.com/paper/2209.02592/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Critical Branching Mechanism in Recurrent Neural Networks](https://mlsanity.com/paper/2606.10384/) (2026)

## Abstract (arXiv)

Deep neural networks and brains both learn and share superficial similarities: processing nodes are likened to neurons and adjustable weights are likened to modifiable synapses. But can a unified theoretical framework be found to underlie them both? Here we show that the equations used to describe neuronal avalanches in living brains can also be applied to cascades of activity in deep neural networks. These equations are derived from non-equilibrium statistical physics and show that deep neural networks learn best when poised between absorbing and active phases. Because these networks are strongly driven by inputs, however, they do not operate at a true critical point but within a quasi-critical regime -- one that still approximately satisfies crackling noise scaling relations. By training networks with different initializations, we show that maximal susceptibility is a more reliable predictor of learning than proximity to the critical point itself. This provides a blueprint for engineering improved network performance. Finally, using finite-size scaling we identify distinct universality classes, including Barkhausen noise and directed percolation. This theoretical framework demonstrates that universal features are shared by both biological and artificial neural networks.