mlsanityData to 14 Oct 2026

Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence

arXiv 2610.18166 ↗First posted 13 Oct 2026stat.ML cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Randomly initialised neural networks of width m converge to Gaussian processes as m → ∞, with two types of quantitative bounds characterising this convergence. Finite-sample bounds compare the network with its Gaussian limit at a fixed finite sample set, on an arbitrary domain, in the quadratic Wasserstein distance mathcal W2, achieving a convergence rate of m-1/2 with explicit constants. However, these constants grow with the number of samples, so the bound becomes uninformative as the sample set is refined. Functional bounds remove that dependence by comparing the two as random functions, at the price of spherical input, metrics weaker than mathcal W2, or slower rates. In this paper, we therefore lift a finite-sample bound to a functional one. The lifting refines the sample set until it is dense in the domain, so the bound must first be made independent of the number of samples. We achieve this by measuring the error per sample, which makes the constant uniform. For a polynomial activation and any compact input set, this gives a rate of m-1/2 in the supremum norm, with an explicit constant.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

22 of 27 references.

References · 22

  1. 2017Deep Neural Networks as Gaussian Processescited by 622
  2. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 566
  3. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  4. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
  5. 2021Dataset Distillation with Infinitely Wide Convolutional Networkscited by 218
  6. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  7. 2019Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processescited by 151
  8. 2020Infinite attention: NNGP and NTK for deep attention networkscited by 121
  9. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 44
  10. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 25
  11. 2023Quantitative CLTs in Deep Neural Networkscited by 20
  12. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 19
10 more
  1. 2021Large-width functional asymptotics for deep Gaussian neural networkscited by 15
  2. 2021Rate of Convergence of Polynomial Networks to Gaussian Processescited by 13
  3. 2023Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processescited by 13
  4. 2023Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalitiescited by 11
  5. 2023Gaussian random field approximation via Stein's method with applications to wide random neural networkscited by 11
  6. 2025Entropic bounds for conditionally Gaussian vectors and applications to neural networkscited by 7
  7. 2023Graph Neural Network-Inspired Kernels for Gaussian Processes in Semi-Supervised Learningcited by 6
  8. 2025Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weightscited by 5
  9. 2026Gaussian Process Limit Reveals Structural Benefits of Graph Transformerscited by 2
  10. 2026Quantitative Gaussian-Process limits of Tensor Programscited by 1

Built on by

Not cited yet.