# Entropic bounds for conditionally Gaussian vectors and applications to neural networks

arXiv: https://arxiv.org/abs/2504.08335 · HTML: https://mlsanity.com/paper/2504.08335/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2025.

## Built on by (most cited first, 7 of 7)

- [Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weights](https://mlsanity.com/paper/2507.12686/) (2025)
- [Universality in Deep Neural Networks: An approach via the Lindeberg exchange principle](https://mlsanity.com/paper/2605.02771/) (2026)
- [Posterior Bayesian Neural Networks with Dependent Weights](https://mlsanity.com/paper/2507.22095/) (2025)
- [Large deviation principles for convolutional Bayesian neural networks](https://mlsanity.com/paper/2603.06023/) (2026)
- [Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputs](https://mlsanity.com/paper/2605.24072/) (2026)
- [Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product](https://mlsanity.com/paper/2606.30831/) (2026)
- [Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence](https://mlsanity.com/paper/2610.18166/) (2026)

## Abstract (arXiv)

Using entropic inequalities from information theory, we provide new bounds on the total variation and 2-Wasserstein distances between a conditionally Gaussian law and a Gaussian law with invertible covariance matrix. We apply our results to quantify the speed of convergence to Gaussian of a randomly initialized fully connected neural network and its derivatives - evaluated in a finite number of inputs - when the initialization is Gaussian and the sizes of the inner layers diverge to infinity. Our results require mild assumptions on the activation function, and allow one to recover optimal rates of convergence in a variety of distances, thus improving and extending the findings of Basteri and Trevisan (2023), Favaro et al. (2023), Trevisan (2024) and Apollonio et al. (2024). One of our main tools are the quantitative cumulant estimates established in Hanin (2024). As an illustration, we apply our results to bound the total variation distance between the Bayesian posterior law of the neural network and its derivatives, and the posterior law of the corresponding Gaussian limit: this yields quantitative versions of a posterior CLT by Hron et al. (2022), and extends several estimates by Trevisan (2024) to the total variation metric.