mlsanityData to 7 Oct 2026

Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networks

arXiv 2203.07379 ↗First posted 14 Mar 2022cs.LG math.PR stat.MLcited by 24

Not on the map. Why?

Abstract (arXiv)

Given any deep fully connected neural network, initialized with random Gaussian parameters, we bound from above the quadratic Wasserstein distance between its output distribution and a suitable Gaussian process. Our explicit inequalities indicate how the hidden and output layers sizes affect the Gaussian behaviour of the network and quantitatively recover the distributional convergence results in the wide limit, i.e., if all the hidden layers sizes become large.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 14 Mar 2022.

Builds on

20 of 41 references.

References · 20

  1. 2018Computational Optimal Transportcited by 765
  2. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  3. 2017Deep Neural Networks as Gaussian Processescited by 617
  4. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  5. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  6. 2021The Principles of Deep Learning Theorycited by 169
  7. 2019Neural Tangents: Fast and Easy Infinite Neural Networks in Pythoncited by 149
  8. 2019Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processescited by 148
  9. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 137
  10. 2020A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networkscited by 68
  11. 2021Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamicscited by 57
  12. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 43
8 more
  1. 2020Exact posterior distributions of wide Bayesian neural networkscited by 29
  2. 2020Stable behaviour of infinitely wide deep neural networkscited by 23
  3. 2020The Ridgelet Prior: A Covariance Function Approach to Prior Specification for Bayesian Neural Networkscited by 20
  4. 2020Wide Neural Networks with Bottlenecks are Deep Gaussian Processescited by 18
  5. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 18
  6. 2021Large-width functional asymptotics for deep Gaussian neural networkscited by 14
  7. 2021Rate of Convergence of Polynomial Networks to Gaussian Processescited by 12
  8. 2018A Gaussian Process perspective on Convolutional Neural Networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 4; 2025: 7; 2026: 5.

24 papers.

Sort:

  1. 2023Quantitative CLTs in Deep Neural Networkscited by 19
  2. 2023Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processescited by 11
  3. 2023Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalitiescited by 10
  4. 2023Gaussian random field approximation via Stein's method with applications to wide random neural networkscited by 10
  5. 2022Gaussian Pre-Activations in Neural Networks: Myth or Reality?cited by 9
  6. 2023A Quantitative Functional Central Limit Theorem for Shallow Neural Networkscited by 8
  7. 2024Spectral complexity of deep neural networkscited by 3
  8. 2024Large Deviations of Gaussian Neural Networks with ReLU activationcited by 3
  9. 2024Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selectioncited by 3
  10. 2025Student-t processes as infinite-width limits of posterior Bayesian neural networkscited by 3
  11. 2025Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weightscited by 3
  12. 2023A note on regularised NTK dynamics with an application to PAC-Bayesian trainingcited by 1
12 more
  1. 2025Fractal and Regular Geometry of Deep Neural Networkscited by 1
  2. 2025Quantitative convergence of trained single layer neural networks to Gaussian processescited by 1
  3. 2022Large-width asymptotics for ReLU neural networks with α-Stable initializationscited by 0
  4. 2024SnatchML: Hijacking ML models without Training Accesscited by 0
  5. 2025Critical Points of Random Neural Networkscited by 0
  6. 2025It Takes a Good Model to Train a Good Model: Generalized Gaussian Priors for Optimized LLMscited by 0
  7. 2025Posterior Bayesian Neural Networks with Dependent Weightscited by 0
  8. 2026Central limit theorems for the outputs of fully convolutional neural networks with time series inputcited by 0
  9. 2026Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputscited by 0
  10. 2026Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Productcited by 0
  11. 2026Quantitative Gaussian-Process limits of Tensor Programscited by 0
  12. 2026High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercubecited by 0