mlsanityData to 14 Oct 2026

Asymptotics of Wide Networks from Feynman Diagrams

arXiv 1909.11304 ↗First posted 25 Sep 2019cs.LG hep-th stat.MLcited by 77

Not on the map. Why?

Abstract (arXiv)

Understanding the asymptotic behavior of wide networks is of considerable interest. In this work, we present a general method for analyzing this large width behavior. The method is an adaptation of Feynman diagrams, a standard tool for computing multivariate Gaussian integrals. We apply our method to study training dynamics, improving existing bounds and deriving new results on wide network evolution during stochastic gradient descent. Going beyond the strict large width limit, we present closed-form expressions for higher-order terms governing wide network training, and test these predictions empirically.

Key facts. Cited by 77 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 18; 2023: 18; 2024: 14; 2025: 10; 2026: 3.

77 papers.

Sort:

  1. 2021Explaining Neural Scaling Lawscited by 363
  2. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  3. 2022Neural Networks can Learn Representations with Gradient Descentcited by 161
  4. 2022High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representationcited by 150
  5. 2022A Solvable Model of Neural Scaling Lawscited by 116
  6. 2024A Dynamical Model of Neural Scaling Lawscited by 111
  7. 2021Neural Networks as Kernel Learners: The Silent Alignment Effectcited by 106
  8. 2022Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networkscited by 105
  9. 2021Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNscited by 53
  10. 2024Scaling and renormalization in high-dimensional regressioncited by 50
  11. 2023The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limitcited by 47
  12. 2023Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networkscited by 43
38 more
  1. 2023Feature-Learning Networks Are Consistent Across Widths At Realistic Scalescited by 41
  2. 2023What can a Single Attention Layer Learn? A Study Through the Random Features Lenscited by 40
  3. 2024Asymptotics of feature learning in two-layer networks after one gradient-stepcited by 38
  4. 2022Limitations of the NTK for Understanding Generalization in Deep Learningcited by 35
  5. 2021A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNscited by 33
  6. 2022Spectral Evolution and Invariance in Linear-width Neural Networkscited by 30
  7. 2021Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlatorscited by 29
  8. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 27
  9. 2021A Theory of Neural Tangent Kernel Alignment and Its Influence on Trainingcited by 25
  10. 2022The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimescited by 25
  11. 2021Nonperturbative renormalization for the neural network-QFT correspondencecited by 24
  12. 2021The edge of chaos: quantum field theory and deep neural networkscited by 24
  13. 2022Contrasting random and learned features in deep Bayesian linear regressioncited by 24
  14. 2022Neural Tangent Kernel: A Surveycited by 23
  15. 2023Neural Network Field Theories: Non-Gaussianity, Actions, and Localitycited by 22
  16. 2023Effective Theory of Transformers at Initializationcited by 21
  17. 2021Understanding Deflation Process in Over-parametrized Tensor Decompositioncited by 20
  18. 2023Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Trainingcited by 20
  19. 2022Meta-Principled Family of Hyperparameter Scaling Strategiescited by 19
  20. 2023Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and widthcited by 19
  21. 2025Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfercited by 19
  22. 2021A theory of representation learning gives a deep generalisation of kernel methodscited by 15
  23. 2021Rate of Convergence of Polynomial Networks to Gaussian Processescited by 13
  24. 2022On Feature Learning in Neural Networks with Global Convergence Guaranteescited by 13
  25. 2023Structures of Neural Network Effective Theoriescited by 13
  26. 2024Towards Understanding Inductive Bias in Transformers: A View From Infinitycited by 13
  27. 2021Reverse Engineering the Neural Tangent Kernelcited by 12
  28. 2022Decomposing neural networks as mappings of correlation functionscited by 12
  29. 2024Coding schemes in neural networks learning classification taskscited by 11
  30. 2022A duality connecting neural network and cosmological dynamicscited by 10
  31. 2022Laziness, Barren Plateau, and Noise in Machine Learningcited by 10
  32. 2025From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learningcited by 10
  33. 2023Depth Separation with Multilayer Mean-Field Networkscited by 9
  34. 2024Bayesian RG Flow in Neural Network Field Theoriescited by 9
  35. 2021Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applicationscited by 8
  36. 2023Catapult Dynamics and Phase Transitions in Quadratic Netscited by 8
  37. 2025Evolution of Concepts in Language Model Pre-Trainingcited by 8
  38. 2025Fermions and Supersymmetry in Neural Network Field Theoriescited by 6

50 most cited of 77 All 77 (JSON)