# Neural Network Field Theories: Non-Gaussianity, Actions, and Locality

arXiv: https://arxiv.org/abs/2307.03223 · HTML: https://mlsanity.com/paper/2307.03223/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 63 references that are in the map.
- First posted on arXiv 6 Jul 2023.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [On Exact Computation with an Infinitely Wide Neural Net](https://mlsanity.com/paper/1904.11955/) (2019)
- [Gaussian Process Behaviour in Wide Deep Neural Networks](https://mlsanity.com/paper/1804.11271/) (2018)
- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels](https://mlsanity.com/paper/1905.13192/) (2019)
- [Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes](https://mlsanity.com/paper/1910.12478/) (2019)
- [Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation](https://mlsanity.com/paper/1902.04760/) (2019)
- [Flow-based generative models for Markov chain Monte Carlo in lattice field theory](https://mlsanity.com/paper/1904.12072/) (2019)
- [Tensor Programs II: Neural Tangent Kernel for Any Architecture](https://mlsanity.com/paper/2006.14548/) (2020)
- [Infinite attention: NNGP and NTK for deep attention networks](https://mlsanity.com/paper/2006.10540/) (2020)
- [Dynamics of Deep Neural Networks and Neural Tangent Hierarchy](https://mlsanity.com/paper/1909.08156/) (2019)
- [Non-Gaussian processes and neural networks at finite widths](https://mlsanity.com/paper/1910.00019/) (2019)
- [Asymptotics of Wide Networks from Feynman Diagrams](https://mlsanity.com/paper/1909.11304/) (2019)
- [The Recurrent Neural Tangent Kernel](https://mlsanity.com/paper/2006.10246/) (2020)
- [Neural Networks and Quantum Field Theory](https://mlsanity.com/paper/2008.08601/) (2020)
- [Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs](https://mlsanity.com/paper/2112.15383/) (2021)
- [Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks](https://mlsanity.com/paper/2304.03408/) (2023)
- [Learning to Unknot](https://mlsanity.com/paper/2010.16263/) (2020)

## Built on by (most cited first, 22 of 22)

- [Rigor with Machine Learning from Field Theory to the Poincar'e Conjecture](https://mlsanity.com/paper/2402.13321/) (2024)
- [Conformal Fields from Neural Networks](https://mlsanity.com/paper/2409.12222/) (2024)
- [On learning higher-order cumulants in diffusion models](https://mlsanity.com/paper/2410.21212/) (2024)
- [Bayesian RG Flow in Neural Network Field Theories](https://mlsanity.com/paper/2405.17538/) (2024)
- [NCoder -- A Quantum Field Theory approach to encoding data](https://mlsanity.com/paper/2402.00944/) (2024)
- [Fermions and Supersymmetry in Neural Network Field Theories](https://mlsanity.com/paper/2511.16741/) (2025)
- [Machine Learning Regularization for the Minimum Volume Formula of Toric Calabi-Yau 3-folds](https://mlsanity.com/paper/2310.19276/) (2023)
- [TASI Lectures on Physics for Machine Learning](https://mlsanity.com/paper/2408.00082/) (2024)
- [Neural network representation of quantum systems](https://mlsanity.com/paper/2403.11420/) (2024)
- [Finite-Width Neural Tangent Kernels from Feynman Diagrams](https://mlsanity.com/paper/2508.11522/) (2025)
- [Conformal Defects in Neural Network Field Theories](https://mlsanity.com/paper/2512.07946/) (2025)
- [Virasoro Symmetry in Neural Network Field Theories](https://mlsanity.com/paper/2512.24420/) (2025)
- [The neural networks with tensor weights and emergent fermionic Wick rules in the large-width limit](https://mlsanity.com/paper/2507.05303/) (2025)
- [Viability of perturbative expansion for quantum field theories on neurons](https://mlsanity.com/paper/2508.03810/) (2025)
- [Topological Effects in Neural Network Field Theory](https://mlsanity.com/paper/2604.02313/) (2026)
- [Phase diagram and eigenvalue dynamics of stochastic gradient descent in multilayer neural networks](https://mlsanity.com/paper/2509.01349/) (2025)
- [Spinning Conformal Correlators from Neural Networks](https://mlsanity.com/paper/2608.15001/) (2026)
- [Bulk-boundary decomposition of neural networks](https://mlsanity.com/paper/2511.02003/) (2025)
- [Neural Network Quantum Field Theory from Transformer Architectures](https://mlsanity.com/paper/2602.10209/) (2026)
- [Deep neural networks as lattice gauge theories](https://mlsanity.com/paper/2608.19331/) (2026)
- [What Neural Network Field Theory Can and Cannot Realise on a Computer](https://mlsanity.com/paper/2608.21523/) (2026)
- [Neural Network Field Theory at Finite Width](https://mlsanity.com/paper/2608.21588/) (2026)

## Abstract (arXiv)

Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. When the central limit theorem can be applied in the infinite-width (infinite-N) limit, the ensemble of networks corresponds to a free field theory. Although an expansion in 1/N corresponds to interactions in the field theory, others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories. These other expansions can be advantageous over the 1/N-expansion, for example by improved behavior with respect to the universal approximation theorem. Given the connected correlators of a field theory, one can systematically reconstruct the action order-by-order in the expansion parameter, using a new Feynman diagram prescription whose vertices are the connected correlators. This method is motivated by the Edgeworth expansion and allows one to derive actions for neural network field theories. Conversely, the correspondence allows one to engineer architectures realizing a given field theory by representing action deformations as deformations of neural network parameter densities. As an example, φ4 theory is realized as an infinite-N neural network field theory.