# The edge of chaos: quantum field theory and deep neural networks

arXiv: https://arxiv.org/abs/2109.13247 · HTML: https://mlsanity.com/paper/2109.13247/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 56 references that are in the map.
- First posted on arXiv 27 Sep 2021.

## Builds on (most cited first, 22 of 22)

- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Exponential expressivity in deep neural networks through transient chaos](https://mlsanity.com/paper/1606.05340/) (2016)
- [Deep Information Propagation](https://mlsanity.com/paper/1611.01232/) (2016)
- [Feature Learning in Infinite-Width Neural Networks](https://mlsanity.com/paper/2011.14522/) (2020)
- [Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks](https://mlsanity.com/paper/1806.05393/) (2018)
- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Non-Gaussian processes and neural networks at finite widths](https://mlsanity.com/paper/1910.00019/) (2019)
- [Asymptotics of Wide Networks from Feynman Diagrams](https://mlsanity.com/paper/1909.11304/) (2019)
- [Neural Networks and Quantum Field Theory](https://mlsanity.com/paper/2008.08601/) (2020)
- [Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks](https://mlsanity.com/paper/1806.05394/) (2018)
- [Noisy Recurrent Neural Networks](https://mlsanity.com/paper/2102.04877/) (2021)
- [On the infinite width limit of neural networks with a standard parameterization](https://mlsanity.com/paper/2001.07301/) (2020)
- [Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlators](https://mlsanity.com/paper/2106.00694/) (2021)
- [Nonperturbative renormalization for the neural network-QFT correspondence](https://mlsanity.com/paper/2108.01403/) (2021)
- [Finite size corrections for neural network Gaussian processes](https://mlsanity.com/paper/1908.10030/) (2019)
- [Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs](https://mlsanity.com/paper/1901.08987/) (2019)
- [Mean-field inference methods for neural networks](https://mlsanity.com/paper/1911.00890/) (2019)
- [Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group](https://mlsanity.com/paper/2107.06898/) (2021)
- [The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning](https://mlsanity.com/paper/2103.04913/) (2021)
- [Why is AI hard and Physics simple?](https://mlsanity.com/paper/2104.00008/) (2021)
- [Controlling a complex system near its critical point via temporal correlations](https://mlsanity.com/paper/1905.11758/) (2019)

## Built on by (most cited first, 24 of 24)

- [Contrasting random and learned features in deep Bayesian linear regression](https://mlsanity.com/paper/2203.00573/) (2022)
- [Unified field theoretical approach to deep and recurrent neuronal networks](https://mlsanity.com/paper/2112.05589/) (2021)
- [Structures of Neural Network Effective Theories](https://mlsanity.com/paper/2305.02334/) (2023)
- [Rigor with Machine Learning from Field Theory to the Poincar'e Conjecture](https://mlsanity.com/paper/2402.13321/) (2024)
- [A duality connecting neural network and cosmological dynamics](https://mlsanity.com/paper/2202.11104/) (2022)
- [Bayesian RG Flow in Neural Network Field Theories](https://mlsanity.com/paper/2405.17538/) (2024)
- [Wilsonian Renormalization of Neural Network Gaussian Processes](https://mlsanity.com/paper/2405.06008/) (2024)
- [Renormalization in the neural network-quantum field theory correspondence](https://mlsanity.com/paper/2212.11811/) (2022)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networks](https://mlsanity.com/paper/2307.02284/) (2023)
- [Neural network representation of quantum systems](https://mlsanity.com/paper/2403.11420/) (2024)
- [Finite-Width Neural Tangent Kernels from Feynman Diagrams](https://mlsanity.com/paper/2508.11522/) (2025)
- [Criticality versus uniformity in deep neural networks](https://mlsanity.com/paper/2304.04784/) (2023)
- [Gauge-covariant stochastic neural fields: Stability and finite-width effects](https://mlsanity.com/paper/2508.18948/) (2025)
- [Virasoro Symmetry in Neural Network Field Theories](https://mlsanity.com/paper/2512.24420/) (2025)
- [Towards Worst-Case Guarantees with Scale-Aware Interpretability](https://mlsanity.com/paper/2602.05184/) (2026)
- [p-Adic Statistical Field Theory and Deep Belief Networks](https://mlsanity.com/paper/2207.13877/) (2022)
- [The neural networks with tensor weights and emergent fermionic Wick rules in the large-width limit](https://mlsanity.com/paper/2507.05303/) (2025)
- [Scale redundancy and soft gauge fixing in positively homogeneous neural networks](https://mlsanity.com/paper/2602.14729/) (2026)
- [Dynamic mean field programming](https://mlsanity.com/paper/2206.05200/) (2022)
- [Black holes and the loss landscape in machine learning](https://mlsanity.com/paper/2306.14817/) (2023)
- [How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrences](https://mlsanity.com/paper/2605.05113/) (2026)
- [Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle](https://mlsanity.com/paper/2605.12536/) (2026)
- [Deep neural networks as lattice gauge theories](https://mlsanity.com/paper/2608.19331/) (2026)

## Abstract (arXiv)

We explicitly construct the quantum field theory corresponding to a general class of deep neural networks encompassing both recurrent and feedforward architectures. We first consider the mean-field theory (MFT) obtained as the leading saddlepoint in the action, and derive the condition for criticality via the largest Lyapunov exponent. We then compute the loop corrections to the correlation function in a perturbative expansion in the ratio of depth T to width N, and find a precise analogy with the well-studied O(N) vector model, in which the variance of the weight initializations plays the role of the 't Hooft coupling. In particular, we compute both the O(1) corrections quantifying fluctuations from typicality in the ensemble of networks, and the subleading O(T/N) corrections due to finite-width effects. These provide corrections to the correlation length that controls the depth to which information can propagate through the network, and thereby sets the scale at which such networks are trainable by gradient descent. Our analysis provides a first-principles approach to the rapidly emerging NN-QFT correspondence, and opens several interesting avenues to the study of criticality in deep neural networks.