# Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights

arXiv: https://arxiv.org/abs/2507.12686 · HTML: https://mlsanity.com/paper/2507.12686/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Builds on (most cited first, 19 of 19)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Gaussian Process Behaviour in Wide Deep Neural Networks](https://mlsanity.com/paper/1804.11271/) (2018)
- [Training Quantized Nets: A Deeper Understanding](https://mlsanity.com/paper/1706.02379/) (2017)
- [Random Neural Networks in the Infinite Width Limit as Gaussian Processes](https://mlsanity.com/paper/2107.01562/) (2021)
- [Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networks](https://mlsanity.com/paper/2203.07379/) (2022)
- [Quantitative CLTs in Deep Neural Networks](https://mlsanity.com/paper/2307.06092/) (2023)
- [Non-asymptotic approximations of neural networks by Gaussian processes](https://mlsanity.com/paper/2102.08668/) (2021)
- [Rate of Convergence of Polynomial Networks to Gaussian Processes](https://mlsanity.com/paper/2111.03175/) (2021)
- [Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility](https://mlsanity.com/paper/2205.08187/) (2022)
- [Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes](https://mlsanity.com/paper/2312.11737/) (2023)
- [Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalities](https://mlsanity.com/paper/2304.04010/) (2023)
- [Gaussian random field approximation via Stein's method with applications to wide random neural networks](https://mlsanity.com/paper/2306.16308/) (2023)
- [Deep Stable neural networks: large-width asymptotics and convergence rates](https://mlsanity.com/paper/2108.02316/) (2021)
- [A Quantitative Functional Central Limit Theorem for Shallow Neural Networks](https://mlsanity.com/paper/2306.16932/) (2023)
- [Entropic bounds for conditionally Gaussian vectors and applications to neural networks](https://mlsanity.com/paper/2504.08335/) (2025)
- [Infinitely wide limits for deep Stable neural networks: sub-linear, linear and super-linear activation functions](https://mlsanity.com/paper/2304.04008/) (2023)
- [Wide neural networks: From non-gaussian random fields at initialization to the NTK geometry of training](https://mlsanity.com/paper/2304.03385/) (2023)
- [Posterior Inference on Shallow Infinitely Wide Bayesian Neural Networks under Weights with Unbounded Variance](https://mlsanity.com/paper/2305.10664/) (2023)

## Built on by (most cited first, 5 of 5)

- [Universality in Deep Neural Networks: An approach via the Lindeberg exchange principle](https://mlsanity.com/paper/2605.02771/) (2026)
- [Posterior Bayesian Neural Networks with Dependent Weights](https://mlsanity.com/paper/2507.22095/) (2025)
- [Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputs](https://mlsanity.com/paper/2605.24072/) (2026)
- [Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product](https://mlsanity.com/paper/2606.30831/) (2026)
- [Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence](https://mlsanity.com/paper/2610.18166/) (2026)

## Abstract (arXiv)

We study the Finite-Dimensional Distributions (FDDs) of deep neural networks with randomly initialized weights that have finite-order moments. Specifically, we establish Gaussian approximation bounds in the Wasserstein-1 norm between the FDDs and their Gaussian limit assuming a Lipschitz activation function and allowing the layer widths to grow to infinity at arbitrary relative rates. In the special case where all widths are proportional to a common scale parameter n and there are L-1 hidden layers, we obtain convergence rates of order n^-(1/6)L-1 + ε, for any ε > 0.