mlsanityData to 14 Oct 2026

Posterior Bayesian Neural Networks with Dependent Weights

arXiv 2507.22095 ↗First posted 29 Jul 2025stat.ML cs.LG math.PRcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider fully connected and feedforward deep neural networks with dependent and possibly heavy-tailed weights, as introduced in [26], to address limitations of the standard Gaussian prior. It has been proved in [26] that, as the number of nodes in the hidden layers grows large, according to a sequential and ordered limit, the law of the output converges weakly to a Gaussian mixture. In this paper, we study the neural network through the lens of the posterior distribution with a Gaussian likelihood. If the random covariance matrix of the infinite-width limit is positive definite under the prior, we identify the posterior distribution of the output in the wide-width limit according to a sequential regime. Remarkably, we provide mild sufficient conditions to ensure the aforementioned invertibility of the random covariance matrix under the prior, thereby extending the results in [8]. Among our results, we present sufficient conditions on some model parameters (the activation function and the associated L'evy measures) which ensure that the sequential limits are independent of the order. We illustrate our findings with examples and numerical simulations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 38 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

24 of 38 references.

References · 24

  1. 2017Deep Neural Networks as Gaussian Processescited by 622
  2. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
  3. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 317
  4. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  5. 2021Bayesian Neural Network Priors Revisitedcited by 121
  6. 2019Traditional and Heavy-Tailed Self Regularization in Neural Network Modelscited by 105
  7. 2021Random Neural Networks in the Infinite Width Limit as Gaussian Processescited by 44
  8. 2020Exact posterior distributions of wide Bayesian neural networkscited by 30
  9. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 25
  10. 2023Quantitative CLTs in Deep Neural Networkscited by 20
  11. 2021Non-asymptotic approximations of neural networks by Gaussian processescited by 19
  12. 2022Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibilitycited by 13
12 more
  1. 2023Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processescited by 13
  2. 2024The Positivity of the Neural Tangent Kernelcited by 13
  3. 2023Non-asymptotic approximations of Gaussian neural networks via second-order Poincar'e inequalitiescited by 11
  4. 2023Gaussian random field approximation via Stein's method with applications to wide random neural networkscited by 11
  5. 2021Deep Stable neural networks: large-width asymptotics and convergence ratescited by 8
  6. 2021Training Integrable Parameterizations of Deep Neural Networks in the Infinite-Width Limitcited by 7
  7. 2025Entropic bounds for conditionally Gaussian vectors and applications to neural networkscited by 7
  8. 2021α-Stable convergence of heavy-tailed infinitely-wide neural networkscited by 5
  9. 2025Finite-Dimensional Gaussian Approximation for Deep Neural Networks: Universality in Random Weightscited by 5
  10. 2024Finite Neural Networks as Mixtures of Gaussian Processes: From Provable Error Bounds to Prior Selectioncited by 3
  11. 2025Student-t processes as infinite-width limits of posterior Bayesian neural networkscited by 3
  12. 2024Function-Space MCMC for Bayesian Wide Neural Networkscited by 1

Built on by

Not cited yet.