mlsanityData to 7 Oct 2026

Rate of Convergence of Polynomial Networks to Gaussian Processes

arXiv 2111.03175 ↗First posted 4 Nov 2021math.PR cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

We examine one-hidden-layer neural networks with random weights. It is well-known that in the limit of infinitely many neurons they simplify to Gaussian processes. For networks with a polynomial activation, we demonstrate that the rate of this convergence in 2-Wasserstein metric is O(n^-frac12), where n is the number of hidden neurons. We suspect this rate is asymptotically sharp. We improve the known convergence rate for other activations, to power-law in n for ReLU and inverse-square-root up to logarithmic factors for erf. We explore the interplay between spherical harmonics, Stein kernels and optimal transport in the non-isotropic setting.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 5 of its 14 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

5 of 14 references.

References · 5

  1. 2019Non-Gaussian processes and neural networks at finite widthscited by 78
  2. 2019Asymptotics of Wide Networks from Feynman Diagramscited by 77
  3. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 43
  4. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 18
  5. 2020On the asymptotics of wide networks with polynomial activationscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 2; 2025: 1; 2026: 1.

12 papers.

  1. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 24
  2. 2023Quantitative CLTs in Deep Neural Networkscited by 19
  3. 2023Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processescited by 11
  4. 2023Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalitiescited by 10
  5. 2023Gaussian random field approximation via Stein's method with applications to wide random neural networkscited by 10
  6. 2023A Quantitative Functional Central Limit Theorem for Shallow Neural Networkscited by 8
  7. 2024Spectral complexity of deep neural networkscited by 3
  8. 2024Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selectioncited by 3
  9. 2025Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weightscited by 3
  10. 2022Convergence of neural networks to Gaussian mixture distributioncited by 1
  11. 2022Large-width asymptotics for ReLU neural networks with α-Stable initializationscited by 0
  12. 2026Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputscited by 0