# Rate of Convergence of Polynomial Networks to Gaussian Processes

arXiv: https://arxiv.org/abs/2111.03175 · HTML: https://mlsanity.com/paper/2111.03175/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 14 references that are in the map.
- First posted on arXiv 4 Nov 2021.

## Builds on (most cited first, 5 of 5)

- [Non-Gaussian processes and neural networks at finite widths](https://mlsanity.com/paper/1910.00019/) (2019)
- [Asymptotics of Wide Networks from Feynman Diagrams](https://mlsanity.com/paper/1909.11304/) (2019)
- [Random Neural Networks in the Infinite Width Limit as Gaussian Processes](https://mlsanity.com/paper/2107.01562/) (2021)
- [Non-asymptotic approximations of neural networks by Gaussian processes](https://mlsanity.com/paper/2102.08668/) (2021)
- [On the asymptotics of wide networks with polynomial activations](https://mlsanity.com/paper/2006.06687/) (2020)

## Built on by (most cited first, 13 of 13)

- [Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networks](https://mlsanity.com/paper/2203.07379/) (2022)
- [Quantitative CLTs in Deep Neural Networks](https://mlsanity.com/paper/2307.06092/) (2023)
- [Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes](https://mlsanity.com/paper/2312.11737/) (2023)
- [Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalities](https://mlsanity.com/paper/2304.04010/) (2023)
- [Gaussian random field approximation via Stein's method with applications to wide random neural networks](https://mlsanity.com/paper/2306.16308/) (2023)
- [A Quantitative Functional Central Limit Theorem for Shallow Neural Networks](https://mlsanity.com/paper/2306.16932/) (2023)
- [Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights](https://mlsanity.com/paper/2507.12686/) (2025)
- [Spectral complexity of deep neural networks](https://mlsanity.com/paper/2405.09541/) (2024)
- [Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selection](https://mlsanity.com/paper/2407.18707/) (2024)
- [Convergence of neural networks to Gaussian mixture distribution](https://mlsanity.com/paper/2204.12100/) (2022)
- [Large-width asymptotics for ReLU neural networks with α-Stable initializations](https://mlsanity.com/paper/2206.08065/) (2022)
- [Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputs](https://mlsanity.com/paper/2605.24072/) (2026)
- [Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence](https://mlsanity.com/paper/2610.18166/) (2026)

## Abstract (arXiv)

We examine one-hidden-layer neural networks with random weights. It is well-known that in the limit of infinitely many neurons they simplify to Gaussian processes. For networks with a polynomial activation, we demonstrate that the rate of this convergence in 2-Wasserstein metric is O(n^-frac12), where n is the number of hidden neurons. We suspect this rate is asymptotically sharp. We improve the known convergence rate for other activations, to power-law in n for ReLU and inverse-square-root up to logarithmic factors for erf. We explore the interplay between spherical harmonics, Stein kernels and optimal transport in the non-isotropic setting.