mlsanityData to 7 Oct 2026

The edge of chaos: quantum field theory and deep neural networks

arXiv 2109.13247 ↗First posted 27 Sep 2021hep-th cond-mat.dis-nn cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

We explicitly construct the quantum field theory corresponding to a general class of deep neural networks encompassing both recurrent and feedforward architectures. We first consider the mean-field theory (MFT) obtained as the leading saddlepoint in the action, and derive the condition for criticality via the largest Lyapunov exponent. We then compute the loop corrections to the correlation function in a perturbative expansion in the ratio of depth T to width N, and find a precise analogy with the well-studied O(N) vector model, in which the variance of the weight initializations plays the role of the 't Hooft coupling. In particular, we compute both the O(1) corrections quantifying fluctuations from typicality in the ensemble of networks, and the subleading O(T/N) corrections due to finite-width effects. These provide corrections to the correlation length that controls the depth to which information can propagate through the network, and thereby sets the scale at which such networks are trainable by gradient descent. Our analysis provides a first-principles approach to the rapidly emerging NN-QFT correspondence, and opens several interesting avenues to the study of criticality in deep neural networks.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 22 of its 56 references that are in the map. First posted on arXiv 27 Sep 2021.

Builds on

22 of 56 references.

References · 22

  1. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  2. 2017Deep Neural Networks as Gaussian Processescited by 617
  3. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  4. 2016Deep Information Propagationcited by 194
  5. 2020Feature Learning in Infinite-Width Neural Networkscited by 182
  6. 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
  7. 2021The Principles of Deep Learning Theorycited by 169
  8. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  9. 2019Asymptotics of Wide Networks from Feynman Diagramscited by 77
  10. 2020Neural Networks and Quantum Field Theorycited by 58
  11. 2018Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networkscited by 44
  12. 2021Noisy Recurrent Neural Networkscited by 42
10 more
  1. 2020On the infinite width limit of neural networks with a standard parameterizationcited by 38
  2. 2021Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlatorscited by 28
  3. 2021Nonperturbative renormalization for the neural network-QFT correspondencecited by 24
  4. 2019Finite size corrections for neural network Gaussian processescited by 22
  5. 2019Dynamical Isometry and a Mean Field Theory of LSTMs and GRUscited by 17
  6. 2019Mean-field inference methods for neural networkscited by 16
  7. 2021Towards quantifying information flows: relative entropy in deep neural networks and the renormalization groupcited by 12
  8. 2021The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learningcited by 9
  9. 2021Why is AI hard and Physics simple?cited by 7
  10. 2019Controlling a complex system near its critical point via temporal correlationscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 4; 2024: 5; 2025: 4; 2026: 4.

23 papers.

Sort:

  1. 2022Contrasting random and learned features in deep Bayesian linear regressioncited by 24
  2. 2021Unified field theoretical approach to deep and recurrent neuronal networkscited by 20
  3. 2023Structures of Neural Network Effective Theoriescited by 13
  4. 2024Rigor with Machine Learning from Field Theory to the Poincar'e Conjecturecited by 12
  5. 2022A duality connecting neural network and cosmological dynamicscited by 10
  6. 2024Bayesian RG Flow in Neural Network Field Theoriescited by 9
  7. 2024Wilsonian Renormalization of Neural Network Gaussian Processescited by 8
  8. 2022Renormalization in the neural network-quantum field theory correspondencecited by 7
  9. 2024Predictive Coding Networks and Inference Learning: Tutorial and Surveycited by 7
  10. 2023Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networkscited by 4
  11. 2024Neural network representation of quantum systemscited by 4
  12. 2023Criticality versus uniformity in deep neural networkscited by 3
11 more
  1. 2025Finite-Width Neural Tangent Kernels from Feynman Diagramscited by 3
  2. 2026Towards Worst-Case Guarantees with Scale-Aware Interpretabilitycited by 3
  3. 2022p-Adic Statistical Field Theory and Deep Belief Networkscited by 2
  4. 2025The neural networks with tensor weights and emergent fermionic Wick rules in the large-width limitcited by 2
  5. 2025Gauge-covariant stochastic neural fields: Stability and finite-width effectscited by 2
  6. 2025Virasoro Symmetry in Neural Network Field Theoriescited by 2
  7. 2026Scale redundancy and soft gauge fixing in positively homogeneous neural networkscited by 1
  8. 2022Dynamic mean field programmingcited by 0
  9. 2023Black holes and the loss landscape in machine learningcited by 0
  10. 2026How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrencescited by 0
  11. 2026Deep neural networks as lattice gauge theoriescited by 0