Convergence of neural networks to Gaussian mixture distribution
Not on the map. Why?
Abstract (arXiv)
We give a proof that, under relatively mild conditions, fully-connected feed-forward deep random neural networks converge to a Gaussian mixture distribution as only the width of the last hidden layer goes to infinity. We conducted experiments for a simple model which supports our result. Moreover, it gives a detailed description of the convergence, namely, the growth of the last hidden layer gets the distribution closer to the Gaussian mixture, and the other layer successively get the Gaussian mixture closer to the normal distribution.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 12 of its 22 references that are in the map. First posted on arXiv 26 Apr 2022.
Builds on
12 of 22 references.
References · 12
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2017Deep Neural Networks as Gaussian Processescited by 617
- 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
- 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 164
- 2019Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processescited by 148
- 2018Deep Convolutional Networks as shallow Gaussian Processescited by 137
- 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 43
- 2020Wide Neural Networks with Bottlenecks are Deep Gaussian Processescited by 18
- 2021Large-width functional asymptotics for deep Gaussian neural networkscited by 14
- 2021Rate of Convergence of Polynomial Networks to Gaussian Processescited by 12
- 2021Deep Stable neural networks: large-width asymptotics and convergence ratescited by 8
Built on by
Citing papers per year: 2023: 1.
1 paper.