mlsanityData to 14 Oct 2026

Finite-Width Neural Tangent Kernels from Feynman Diagrams

arXiv 2508.11522 ↗First posted 15 Aug 2025cs.LG hep-thcited by 4

Not on the map. Why?

Abstract (arXiv)

Neural tangent kernels (NTKs) are a powerful tool for analyzing deep, non-linear neural networks. In the infinite-width limit, NTKs can easily be computed for most common architectures, yielding full analytic control over the training dynamics. However, at infinite width, important properties of training such as NTK evolution or feature learning are absent. Nevertheless, finite width effects can be included by computing corrections to the Gaussian statistics at infinite width. We introduce Feynman diagrams for computing finite-width corrections to NTK statistics. These dramatically simplify the necessary algebraic manipulations and enable the computation of layer-wise recursion relations for arbitrary statistics involving preactivations, NTKs and certain higher-derivative tensors (dNTK and ddNTK) required to predict the training dynamics at leading order. We demonstrate the feasibility of our framework by extending stability results for deep networks from preactivations to NTKs and proving the absence of finite-width corrections for scale-invariant nonlinearities such as ReLU on the diagonal of the Gram matrix of the NTK. We numerically implement the complete set of equations necessary to compute the first-order corrections for arbitrary inputs and demonstrate that the results follow the statistics of sampled neural networks for widths ngtrsim 20.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 15 Aug 2025.

Builds on

33 of 47 references.

References · 33

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  4. 2017Deep Neural Networks as Gaussian Processescited by 622
  5. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  6. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  7. 2020Finite Versus Infinite Neural Networks: an Empirical Studycited by 157
  8. 2019Neural Tangents: Fast and Easy Infinite Neural Networks in Pythoncited by 149
  9. 2022A Solvable Model of Neural Scaling Lawscited by 116
  10. 2019Finite Depth and Width Corrections to the Neural Tangent Kernelcited by 115
  11. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  12. 2019Scaling description of generalization with number of parameters in deep learningcited by 83
21 more
  1. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  2. 2019Asymptotics of Wide Networks from Feynman Diagramscited by 77
  3. 2020Neural Networks and Quantum Field Theorycited by 61
  4. 2022Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initializationcited by 30
  5. 2021Symmetry-via-Duality: Invariant Neural Network Densities from Parameter-Space Correlatorscited by 29
  6. 2021Nonperturbative renormalization for the neural network-QFT correspondencecited by 24
  7. 2021The edge of chaos: quantum field theory and deep neural networkscited by 24
  8. 2023Neural Network Field Theories: Non-Gaussianity, Actions, and Localitycited by 22
  9. 2021A Neural Tangent Kernel Perspective of GANscited by 21
  10. 2022Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?cited by 18
  11. 2024Emergent Equivariance in Deep Ensemblescited by 18
  12. 2022Few-shot Backdoor Attacks via Neural Tangent Kernelscited by 15
  13. 2023Structures of Neural Network Effective Theoriescited by 13
  14. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  15. 2023Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate Modelscited by 8
  16. 2023On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspectivecited by 6
  17. 2023Feature Learning and Generalization in Deep Networks with Orthogonal Weightscited by 5
  18. 2024Neural Scaling Laws From Large-N Field Theory: Solvable Model Beyond the Ridgeless Limitcited by 5
  19. 2024Equivariant Neural Tangent Kernelscited by 4
  20. 2024When Training-Free NAS Meets Vision Transformer: A Neural Tangent Kernel Perspectivecited by 2
  21. 2025Issues with Neural Tangent Kernel Approach to Neural Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoningcited by 0
  2. 2026Loop Corrections in Random Feature Models: Training Error and Generalization Gapcited by 0
  3. 2026Collective Kernel EFT for Pre-activation ResNetscited by 0
  4. 2026Criticality and Saturation in Orthogonal Neural Networkscited by 0