# Unified field theoretical approach to deep and recurrent neuronal networks

arXiv: https://arxiv.org/abs/2112.05589 · HTML: https://mlsanity.com/paper/2112.05589/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 9 of 9)

- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization](https://mlsanity.com/paper/2012.04030/) (2020)
- [Neural Networks and Quantum Field Theory](https://mlsanity.com/paper/2008.08601/) (2020)
- [Predicting the outputs of finite deep neural networks trained with noisy gradients](https://mlsanity.com/paper/2004.01190/) (2020)
- [The edge of chaos: quantum field theory and deep neural networks](https://mlsanity.com/paper/2109.13247/) (2021)
- [Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks](https://mlsanity.com/paper/1707.08337/) (2017)
- [A Microscopic Theory of Intrinsic Timescales in Spiking Neural Networks](https://mlsanity.com/paper/1909.01908/) (2019)
- [Space of Functions Computed by Deep-Layered Machines](https://mlsanity.com/paper/2004.08930/) (2020)
- [Large Deviations Approach to Random Recurrent Neuronal Networks: Parameter Inference and Fluctuation-Induced Transitions](https://mlsanity.com/paper/2009.08889/) (2020)

## Built on by (most cited first, 20 of 20)

- [Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks](https://mlsanity.com/paper/2205.09653/) (2022)
- [Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit](https://mlsanity.com/paper/2309.16620/) (2023)
- [Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks](https://mlsanity.com/paper/2304.03408/) (2023)
- [There Will Be a Scientific Theory of Deep Learning](https://mlsanity.com/paper/2604.21691/) (2026)
- [Structures of Neural Network Effective Theories](https://mlsanity.com/paper/2305.02334/) (2023)
- [Towards Understanding Inductive Bias in Transformers: A View From Infinity](https://mlsanity.com/paper/2402.05173/) (2024)
- [Decomposing neural networks as mappings of correlation functions](https://mlsanity.com/paper/2202.04925/) (2022)
- [Coding schemes in neural networks learning classification tasks](https://mlsanity.com/paper/2406.16689/) (2024)
- [From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning](https://mlsanity.com/paper/2502.03210/) (2025)
- [Field theory for optimal signal propagation in ResNets](https://mlsanity.com/paper/2305.07715/) (2023)
- [Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings](https://mlsanity.com/paper/2305.08459/) (2023)
- [Eight challenges in developing theory of intelligence](https://mlsanity.com/paper/2306.11232/) (2023)
- [A theory of data variability in Neural Network Bayesian inference](https://mlsanity.com/paper/2307.16695/) (2023)
- [A unified theory of feature learning in RNNs and DNNs](https://mlsanity.com/paper/2602.15593/) (2026)
- [Deep Neural Networks: A Formulation Via Non-Archimedean Analysis](https://mlsanity.com/paper/2402.00094/) (2024)
- [Dynamic mean field programming](https://mlsanity.com/paper/2206.05200/) (2022)
- [Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropy](https://mlsanity.com/paper/2510.04285/) (2025)
- [Bulk-boundary decomposition of neural networks](https://mlsanity.com/paper/2511.02003/) (2025)
- [How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrences](https://mlsanity.com/paper/2605.05113/) (2026)
- [Deep neural networks as lattice gauge theories](https://mlsanity.com/paper/2608.19331/) (2026)

## Abstract (arXiv)

Understanding capabilities and limitations of different network architectures is of fundamental importance to machine learning. Bayesian inference on Gaussian processes has proven to be a viable approach for studying recurrent and deep networks in the limit of infinite layer width, n→∞. Here we present a unified and systematic derivation of the mean-field theory for both architectures that starts from first principles by employing established methods from statistical physics of disordered systems. The theory elucidates that while the mean-field equations are different with regard to their temporal structure, they yet yield identical Gaussian kernels when readouts are taken at a single time point or layer, respectively. Bayesian inference applied to classification then predicts identical performance and capabilities for the two architectures. Numerically, we find that convergence towards the mean-field theory is typically slower for recurrent networks than for deep networks and the convergence speed depends non-trivially on the parameters of the weight prior as well as the depth or number of time steps, respectively. Our method exposes that Gaussian processes are but the lowest order of a systematic expansion in 1/n and we compute next-to-leading-order corrections which turn out to be architecture-specific. The formalism thus paves the way to investigate the fundamental differences between recurrent and deep architectures at finite widths n.