mlsanityData to 14 Oct 2026

Quantitative convergence of trained single layer neural networks to Gaussian processes

arXiv 2509.24544 ↗First posted 29 Sep 2025stat.ML cs.LG math.PRcited by 1

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the quantitative convergence of shallow neural networks trained via gradient descent to their associated Gaussian processes in the infinite-width limit. While previous work has established qualitative convergence under broad settings, precise, finite-width estimates remain limited, particularly during training. We provide explicit upper bounds on the quadratic Wasserstein distance between the network output and its Gaussian approximation at any training time t ≥ 0, demonstrating polynomial decay with network width. Our results quantify how architectural parameters, such as width and input dimension, influence convergence, and how training dynamics affect the approximation error.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

26 of 38 references.

References · 26

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  4. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  5. 2018On Lazy Training in Differentiable Programmingcited by 675
  6. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  7. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 566
  8. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  9. 2021Deep learning: a statistical viewpointcited by 240
  10. 2019Neural Tangents: Fast and Easy Infinite Neural Networks in Pythoncited by 149
  11. 2018Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernelcited by 131
  12. 2019An Improved Analysis of Training Over-parameterized Deep Neural Networkscited by 121
14 more
  1. 2019Dynamics of Deep Neural Networks and Neural Tangent Hierarchycited by 96
  2. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  3. 2021Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamicscited by 57
  4. 2019The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descentcited by 46
  5. 2022Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterizationcited by 36
  6. 2019Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Networkcited by 27
  7. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 25
  8. 2023Quantitative CLTs in Deep Neural Networkscited by 20
  9. 2021Smooth p-Wasserstein Distance: Structure, Empirical Approximation, and Statistical Applicationscited by 15
  10. 2021Controlling Wasserstein Distances by Kernel Norms with Application to Compressive Statistical Learningcited by 14
  11. 2023Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processescited by 13
  12. 2024The Positivity of the Neural Tangent Kernelcited by 13
  13. 2024Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimensioncited by 13
  14. 2023Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalitiescited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Quantitative Gaussian-Process limits of Tensor Programscited by 1