mlsanityData to 14 Oct 2026

Collective Kernel EFT for Pre-activation ResNets

arXiv 2604.15742 ↗First posted 17 Apr 2026cs.LG hep-th stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

In finite-width deep neural networks, the empirical kernel G evolves stochastically across layers. We develop a collective kernel effective field theory (EFT) for pre-activation ResNets based on a G-only closure hierarchy and diagnose its finite validity window. Exploiting the exact conditional Gaussianity of residual increments, we derive an exact stochastic recursion for G. Applying Gaussian approximations systematically yields a continuous-depth ODE system for the mean kernel K0, the kernel covariance V4, and the 1/n mean correction K1,EFT, which emerges diagrammatically as a one-loop tadpole correction. Numerically, K0 remains accurate at all depths. However, the V4 equation residual accumulates to an O(1) error at finite time, primarily driven by approximation errors in the G-only transport term. Furthermore, K1,EFT fails due to the breakdown of the source closure, which exhibits a systematic mismatch even at initialization. These findings highlight the limitations of G-only state-space reduction and suggest extending the state space to incorporate the sigma-kernel.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 26 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

20 of 26 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  4. 2017Deep Neural Networks as Gaussian Processescited by 622
  5. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  6. 2016Exponential expressivity in deep neural networks through transient chaoscited by 261
  7. 2016Deep Information Propagationcited by 195
  8. 2021The Principles of Deep Learning Theorycited by 173
  9. 2017Mean Field Residual Networks: On the Edge of Chaoscited by 83
  10. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  11. 2020Stable ResNetcited by 49
  12. 2022The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initializationcited by 41
8 more
  1. 2021The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initializationcited by 33
  2. 2022Random Fully Connected Neural Networks as Perturbatively Solvable Hierarchiescited by 24
  3. 2019Infinitely deep neural networks as diffusion processescited by 21
  4. 2023Width and Depth Limits Commute in Residual Networkscited by 21
  5. 2023Structures of Neural Network Effective Theoriescited by 13
  6. 2023Field theory for optimal signal propagation in ResNetscited by 6
  7. 2020Doubly infinite residual neural networks: a diffusion process approachcited by 4
  8. 2025Finite-Width Neural Tangent Kernels from Feynman Diagramscited by 4

Built on by

Not cited yet.