mlsanityData to 7 Oct 2026

Structures of Neural Network Effective Theories

arXiv 2305.02334 ↗First posted 3 May 2023hep-th cond-mat.dis-nn cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

36 of 46 references.

References · 36

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  3. 2017Deep Neural Networks as Gaussian Processescited by 617
  4. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  5. 2016On the Expressive Power of Deep Neural Networkscited by 302
  6. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  7. 2016Deep Information Propagationcited by 194
  8. 2021The Principles of Deep Learning Theorycited by 169
  9. 2019Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processescited by 148
  10. 2020Tensor Programs II: Neural Tangent Kernel for Any Architecturecited by 139
  11. 2019Finite Depth and Width Corrections to the Neural Tangent Kernelcited by 114
  12. 2022A Solvable Model of Neural Scaling Lawscited by 101
24 more
  1. 2019Dynamics of Deep Neural Networks and Neural Tangent Hierarchycited by 96
  2. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  3. 2019Asymptotics of Wide Networks from Feynman Diagramscited by 77
  4. 2020Neural Networks and Quantum Field Theorycited by 58
  5. 2021Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNscited by 53
  6. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 43
  7. 2021A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNscited by 33
  8. 2020Predicting the outputs of finite deep neural networks trained with noisy gradientscited by 31
  9. 2021Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlatorscited by 28
  10. 2021Precise characterization of the prior predictive distribution of deep ReLU networkscited by 28
  11. 2021Building Quantum Field Theories Out of Neuronscited by 27
  12. 2021Nonperturbative renormalization for the neural network-QFT correspondencecited by 24
  13. 2021The edge of chaos: quantum field theory and deep neural networkscited by 23
  14. 2019Finite size corrections for neural network Gaussian processescited by 22
  15. 2022Bayesian Interpolation with Deep Linear Networkscited by 21
  16. 2023Effective Theory of Transformers at Initializationcited by 21
  17. 2021Unified field theoretical approach to deep and recurrent neuronal networkscited by 20
  18. 2022Meta-Principled Family of Hyperparameter Scaling Strategiescited by 18
  19. 2020On the asymptotics of wide networks with polynomial activationscited by 17
  20. 2021Quantum field-theoretic machine learningcited by 16
  21. 2020Asymptotics of Wide Convolutional Neural Networkscited by 15
  22. 2017A Correspondence Between Random Neural Networks and Statistical Field Theorycited by 13
  23. 2021Why is AI hard and Physics simple?cited by 7
  24. 2022Renormalization in the neural network-quantum field theory correspondencecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 4; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Neural Network Field Theories: Non-Gaussianity, Actions, and Localitycited by 21
  2. 2024Rigor with Machine Learning from Field Theory to the Poincar'e Conjecturecited by 12
  3. 2024Bayesian RG Flow in Neural Network Field Theoriescited by 9
  4. 2024Bayesian Inference with Deep Weakly Nonlinear Networkscited by 5
  5. 2025Fermions and Supersymmetry in Neural Network Field Theoriescited by 5
  6. 2024Equivariant Neural Tangent Kernelscited by 4
  7. 2025Finite-Width Neural Tangent Kernels from Feynman Diagramscited by 3
  8. 2024Why Rectified Power Unit Networks Fail and How to Improve It: An Effective Field Theory Perspectivecited by 1
  9. 2025Viability of perturbative expansion for quantum field theories on neuronscited by 1
  10. 2023Black holes and the loss landscape in machine learningcited by 0
  11. 2025Analysis of Fourier Neural Operators via Effective Field Theorycited by 0
  12. 2026FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoningcited by 0
1 more
  1. 2026Collective Kernel EFT for Pre-activation ResNetscited by 0