# Nonperturbative renormalization for the neural network-QFT correspondence

arXiv: https://arxiv.org/abs/2108.01403 · HTML: https://mlsanity.com/paper/2108.01403/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 102 references that are in the map.
- First posted on arXiv 3 Aug 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Gaussian Process Behaviour in Wide Deep Neural Networks](https://mlsanity.com/paper/1804.11271/) (2018)
- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes](https://mlsanity.com/paper/1810.05148/) (2018)
- [Explainable Machine Learning for Scientific Insights and Discoveries](https://mlsanity.com/paper/1905.08883/) (2019)
- [Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes](https://mlsanity.com/paper/1910.12478/) (2019)
- [Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation](https://mlsanity.com/paper/1902.04760/) (2019)
- [Tensor Programs II: Neural Tangent Kernel for Any Architecture](https://mlsanity.com/paper/2006.14548/) (2020)
- [Deep Convolutional Networks as shallow Gaussian Processes](https://mlsanity.com/paper/1808.05587/) (2018)
- [Non-Gaussian processes and neural networks at finite widths](https://mlsanity.com/paper/1910.00019/) (2019)
- [Asymptotics of Wide Networks from Feynman Diagrams](https://mlsanity.com/paper/1909.11304/) (2019)
- [Neural Networks and Quantum Field Theory](https://mlsanity.com/paper/2008.08601/) (2020)
- [An exact mapping between the Variational Renormalization Group and Deep Learning](https://mlsanity.com/paper/1410.3831/) (2014)
- [Tensor Programs III: Neural Matrix Laws](https://mlsanity.com/paper/2009.10685/) (2020)
- [Neural Network Renormalization Group](https://mlsanity.com/paper/1802.02840/) (2018)
- [The Challenge of Crafting Intelligible Intelligence](https://mlsanity.com/paper/1803.04263/) (2018)
- [Detecting Symmetries with Neural Networks](https://mlsanity.com/paper/2003.13679/) (2020)
- [Mutual Information, Neural Networks and the Renormalization Group](https://mlsanity.com/paper/1704.06279/) (2017)
- [Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlators](https://mlsanity.com/paper/2106.00694/) (2021)
- [Quantum field-theoretic machine learning](https://mlsanity.com/paper/2102.09449/) (2021)
- [A Correspondence Between Random Neural Networks and Statistical Field Theory](https://mlsanity.com/paper/1710.06570/) (2017)
- [PCA meets RG](https://mlsanity.com/paper/1610.09733/) (2016)
- [Connecting Dualities and Machine Learning](https://mlsanity.com/paper/2002.05169/) (2020)
- [Why Unsupervised Deep Networks Generalize](https://mlsanity.com/paper/2012.03531/) (2020)
- [Inferring relevant features: from QFT to PCA](https://mlsanity.com/paper/1802.05756/) (2018)
- [Short sighted deep learning](https://mlsanity.com/paper/2002.02664/) (2020)

## Built on by (most cited first, 24 of 24)

- [The edge of chaos: quantum field theory and deep neural networks](https://mlsanity.com/paper/2109.13247/) (2021)
- [Neural Network Field Theories: Non-Gaussianity, Actions, and Locality](https://mlsanity.com/paper/2307.03223/) (2023)
- [Emergent Equivariance in Deep Ensembles](https://mlsanity.com/paper/2403.03103/) (2024)
- [Structures of Neural Network Effective Theories](https://mlsanity.com/paper/2305.02334/) (2023)
- [Rigor with Machine Learning from Field Theory to the Poincar'e Conjecture](https://mlsanity.com/paper/2402.13321/) (2024)
- [A duality connecting neural network and cosmological dynamics](https://mlsanity.com/paper/2202.11104/) (2022)
- [Bayesian RG Flow in Neural Network Field Theories](https://mlsanity.com/paper/2405.17538/) (2024)
- [Wilsonian Renormalization of Neural Network Gaussian Processes](https://mlsanity.com/paper/2405.06008/) (2024)
- [Renormalization in the neural network-quantum field theory correspondence](https://mlsanity.com/paper/2212.11811/) (2022)
- [Fermions and Supersymmetry in Neural Network Field Theories](https://mlsanity.com/paper/2511.16741/) (2025)
- [A Triumvirate of AI Driven Theoretical Discovery](https://mlsanity.com/paper/2405.19973/) (2024)
- [Equivariant Neural Tangent Kernels](https://mlsanity.com/paper/2406.06504/) (2024)
- [Finite-Width Neural Tangent Kernels from Feynman Diagrams](https://mlsanity.com/paper/2508.11522/) (2025)
- [Gauge-covariant stochastic neural fields: Stability and finite-width effects](https://mlsanity.com/paper/2508.18948/) (2025)
- [p-Adic Statistical Field Theory and Deep Belief Networks](https://mlsanity.com/paper/2207.13877/) (2022)
- [The neural networks with tensor weights and emergent fermionic Wick rules in the large-width limit](https://mlsanity.com/paper/2507.05303/) (2025)
- [Viability of perturbative expansion for quantum field theories on neurons](https://mlsanity.com/paper/2508.03810/) (2025)
- [Scale redundancy and soft gauge fixing in positively homogeneous neural networks](https://mlsanity.com/paper/2602.14729/) (2026)
- [Phase diagram and eigenvalue dynamics of stochastic gradient descent in multilayer neural networks](https://mlsanity.com/paper/2509.01349/) (2025)
- [Characterizing 4-string contact interaction using machine learning](https://mlsanity.com/paper/2211.09129/) (2022)
- [Black holes and the loss landscape in machine learning](https://mlsanity.com/paper/2306.14817/) (2023)
- [Bulk-boundary decomposition of neural networks](https://mlsanity.com/paper/2511.02003/) (2025)
- [Deep neural networks as lattice gauge theories](https://mlsanity.com/paper/2608.19331/) (2026)
- [Pre-registered tests of solid-state-physics-inspired LLM compression: a cluster-level negative result at small-language-model scale](https://mlsanity.com/paper/2609.34292/) (2026)

## Abstract (arXiv)

In a recent work arXiv:2008.08601, Halverson, Maiti and Stoner proposed a description of neural networks in terms of a Wilsonian effective field theory. The infinite-width limit is mapped to a free field theory, while finite N corrections are taken into account by interactions (non-Gaussian terms in the action). In this paper, we study two related aspects of this correspondence. First, we comment on the concepts of locality and power-counting in this context. Indeed, these usual space-time notions may not hold for neural networks (since inputs can be arbitrary), however, the renormalization group provides natural notions of locality and scaling. Moreover, we comment on several subtleties, for example, that data components may not have a permutation symmetry: in that case, we argue that random tensor field theories could provide a natural generalization. Second, we improve the perturbative Wilsonian renormalization from arXiv:2008.08601 by providing an analysis in terms of the nonperturbative renormalization group using the Wetterich-Morris equation. An important difference with usual nonperturbative RG analysis is that only the effective (IR) 2-point function is known, which requires setting the problem with care. Our aim is to provide a useful formalism to investigate neural networks behavior beyond the large-width limit (i.e.~far from Gaussian limit) in a nonperturbative fashion. A major result of our analysis is that changing the standard deviation of the neural network weight distribution can be interpreted as a renormalization flow in the space of networks. We focus on translations invariant kernels and provide preliminary numerical results.