# Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalities

arXiv: https://arxiv.org/abs/2304.04010 · HTML: https://mlsanity.com/paper/2304.04010/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 8 Apr 2023.

## Builds on (most cited first, 16 of 16)

- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Gaussian Process Behaviour in Wide Deep Neural Networks](https://mlsanity.com/paper/1804.11271/) (2018)
- [Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes](https://mlsanity.com/paper/1810.05148/) (2018)
- [Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes](https://mlsanity.com/paper/1910.12478/) (2019)
- [Deep Convolutional Networks as shallow Gaussian Processes](https://mlsanity.com/paper/1808.05587/) (2018)
- [Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations](https://mlsanity.com/paper/1708.02691/) (2017)
- [Random Neural Networks in the Infinite Width Limit as Gaussian Processes](https://mlsanity.com/paper/2107.01562/) (2021)
- [Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networks](https://mlsanity.com/paper/2203.07379/) (2022)
- [Finite size corrections for neural network Gaussian processes](https://mlsanity.com/paper/1908.10030/) (2019)
- [Non-asymptotic approximations of neural networks by Gaussian processes](https://mlsanity.com/paper/2102.08668/) (2021)
- [On the asymptotics of wide networks with polynomial activations](https://mlsanity.com/paper/2006.06687/) (2020)
- [Asymptotics of Wide Convolutional Neural Networks](https://mlsanity.com/paper/2008.08675/) (2020)
- [Large-width functional asymptotics for deep Gaussian neural networks](https://mlsanity.com/paper/2102.10307/) (2021)
- [Rate of Convergence of Polynomial Networks to Gaussian Processes](https://mlsanity.com/paper/2111.03175/) (2021)
- [Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility](https://mlsanity.com/paper/2205.08187/) (2022)
- [Deep Stable neural networks: large-width asymptotics and convergence rates](https://mlsanity.com/paper/2108.02316/) (2021)

## Built on by (most cited first, 11 of 11)

- [Quantitative CLTs in Deep Neural Networks](https://mlsanity.com/paper/2307.06092/) (2023)
- [Gaussian random field approximation via Stein's method with applications to wide random neural networks](https://mlsanity.com/paper/2306.16308/) (2023)
- [A Quantitative Functional Central Limit Theorem for Shallow Neural Networks](https://mlsanity.com/paper/2306.16932/) (2023)
- [Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights](https://mlsanity.com/paper/2507.12686/) (2025)
- [Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selection](https://mlsanity.com/paper/2407.18707/) (2024)
- [Quantitative convergence of trained single layer neural networks to Gaussian processes](https://mlsanity.com/paper/2509.24544/) (2025)
- [Quantitative Gaussian-Process limits of Tensor Programs](https://mlsanity.com/paper/2607.06290/) (2026)
- [Posterior Bayesian Neural Networks with Dependent Weights](https://mlsanity.com/paper/2507.22095/) (2025)
- [Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputs](https://mlsanity.com/paper/2605.24072/) (2026)
- [Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product](https://mlsanity.com/paper/2606.30831/) (2026)
- [Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence](https://mlsanity.com/paper/2610.18166/) (2026)

## Abstract (arXiv)

There is a recent and growing literature on large-width asymptotic and non-asymptotic properties of deep Gaussian neural networks (NNs), namely NNs with weights initialized as Gaussian distributions. For a Gaussian NN of depth L≥1 and width n≥1, it is well-known that, as nrightarrow+∞, the NN's output converges (in distribution) to a Gaussian process. Recently, some quantitative versions of this result, also known as quantitative central limit theorems (QCLTs), have been obtained, showing that the rate of convergence is n-1, in the 2-Wasserstein distance, and that such a rate is optimal. In this paper, we investigate the use of second-order Poincar'e inequalities as an alternative approach to establish QCLTs for the NN's output. Previous approaches consist of a careful analysis of the NN, by combining non-trivial probabilistic tools with ad-hoc techniques that rely on the recursive definition of the network, typically by means of an induction argument over the layers, and it is unclear if and how they still apply to other NN's architectures. Instead, the use of second-order Poincar'e inequalities rely only on the fact that the NN is a functional of a Gaussian process, reducing the problem of establishing QCLTs to the algebraic problem of computing the gradient and Hessian of the NN's output, which still applies to other NN's architectures. We show how our approach is effective in establishing QCLTs for the NN's output, though it leads to suboptimal rates of convergence. We argue that such a worsening in the rates is peculiar to second-order Poincar'e inequalities, and it should be interpreted as the "cost" for having a straightforward, and general, procedure for obtaining QCLTs.