mlsanityData to 14 Oct 2026

Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes

arXiv 2312.11737 ↗First posted 18 Dec 2023math.ST math.PR stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

We establish novel rates for the Gaussian approximation of random deep neural networks with Gaussian parameters (weights and biases) and Lipschitz activation functions, in the wide limit. Our bounds apply for the joint output of a network evaluated any finite input set, provided a certain non-degeneracy condition of the infinite-width covariances holds. We demonstrate that the distance between the network output and the corresponding Gaussian approximation scales inversely with the width of the network, exhibiting faster convergence than the naive heuristic suggested by the central limit theorem. We also apply our bounds to obtain theoretical approximations for the exact Bayesian posterior distribution of the network, when the likelihood is a bounded Lipschitz function of the network output evaluated on a (finite) training set. This includes popular cases such as the Gaussian likelihood, i.e. exponential of minus the mean squared error.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 13 of its 31 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

13 of 31 references.

References · 13

  1. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  2. 2017Deep Neural Networks as Gaussian Processescited by 622
  3. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  4. 2020A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networkscited by 68
  5. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 44
  6. 2020Exact posterior distributions of wide Bayesian neural networkscited by 30
  7. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 25
  8. 2023Quantitative CLTs in Deep Neural Networkscited by 20
  9. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 19
  10. 2021Large-width functional asymptotics for deep Gaussian neural networkscited by 15
  11. 2021Rate of Convergence of Polynomial Networks to Gaussian Processescited by 13
  12. 2023Gaussian random field approximation via Stein's method with applications to wide random neural networkscited by 11
1 more
  1. 2023A Quantitative Functional Central Limit Theorem for Shallow Neural Networkscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 7.

13 papers.

Sort:

  1. 2024Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layerscited by 12
  2. 2025Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weightscited by 5
  3. 2024Proportional infinite-width infinite-depth limit for deep linear neural networkscited by 3
  4. 2025Student-t processes as infinite-width limits of posterior Bayesian neural networkscited by 3
  5. 2026Universality in Deep Neural Networks: An approach via the Lindeberg exchange principlecited by 2
  6. 2025Quantitative convergence of trained single layer neural networks to Gaussian processescited by 1
  7. 2026Bayesian Inference with Shaped Deep Non-linear MLPscited by 1
  8. 2026Quantitative Gaussian-Process limits of Tensor Programscited by 1
  9. 2025Posterior Bayesian Neural Networks with Dependent Weightscited by 0
  10. 2026Large deviation principles for convolutional Bayesian neural networkscited by 0
  11. 2026Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputscited by 0
  12. 2026Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Productcited by 0
1 more
  1. 2026Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergencecited by 0