mlsanityData to 14 Oct 2026

Exact posterior distributions of wide Bayesian neural networks

arXiv 2006.10541 ↗First posted 18 Jun 2020stat.ML cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that the prior over functions induced by a deep Bayesian neural network (BNN) behaves as a Gaussian process (GP) as the width of all layers becomes large. However, many BNN applications are concerned with the BNN function space posterior. While some empirical evidence of the posterior convergence was provided in the original works of Neal (1996) and Matthews et al. (2018), it is limited to small datasets or architectures due to the notorious difficulty of obtaining and verifying exactness of BNN posterior approximations. We provide the missing theoretical proof that the exact BNN posterior converges (weakly) to the one induced by the GP limit of the prior. For empirical validation, we show how to generate exact samples from a finite BNN on a small dataset via rejection sampling.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 10; 2023: 4; 2024: 3; 2025: 4; 2026: 3.

30 papers.

Sort:

  1. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  2. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  3. 2022Fast Finite Width Neural Tangent Kernelcited by 62
  4. 2023Bayes-optimal Learning of Deep Random Networks of Extensive-widthcited by 41
  5. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 27
  6. 2022Quantitative Gaussian Approximation of Randomly Initialized Deep Neural Networkscited by 25
  7. 2022Contrasting random and learned features in deep Bayesian linear regressioncited by 24
  8. 2022Wide Mean-Field Bayesian Neural Networks Ignore the Datacited by 22
  9. 2021The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspectivecited by 21
  10. 2022The Influence of Learning Rule on Representation Dynamics in Wide Neural Networkscited by 19
  11. 2022Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimizationcited by 18
  12. 2021A theory of representation learning gives a deep generalisation of kernel methodscited by 15
18 more
  1. 2023Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processescited by 13
  2. 2022On the Robustness of Bayesian Neural Networks to Adversarial Attackscited by 12
  3. 2022Wide Bayesian neural networks have a simple weight posterior: theory and accelerated samplingcited by 9
  4. 2024Bayesian RG Flow in Neural Network Field Theoriescited by 9
  5. 2021Wide Mean-Field Variational Bayesian Neural Networks Ignore the Datacited by 7
  6. 2021Depth induces scale-averaging in overparameterized linear Bayesian neural networkscited by 7
  7. 2022Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithmcited by 6
  8. 2025Student-t processes as infinite-width limits of posterior Bayesian neural networkscited by 3
  9. 2022On neural network kernels and the storage capacity problemcited by 2
  10. 2024Revisiting the Equivalence of Bayesian Neural Networks and Gaussian Processes: On the Importance of Learning Activationscited by 2
  11. 2025Unreliable Uncertainty Estimates with Monte Carlo Dropoutcited by 2
  12. 2024Function-Space MCMC for Bayesian Wide Neural Networkscited by 1
  13. 2023Sparsity-depth Tradeoff in Infinitely Wide Deep Neural Networkscited by 0
  14. 2025Posterior Bayesian Neural Networks with Dependent Weightscited by 0
  15. 2025Hi-fi functional priors by learning activationscited by 0
  16. 2026Bayesian Interpolating Neural Network (B-INN): a scalable and reliable Bayesian model for large-scale physical systemscited by 0
  17. 2026Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputscited by 0
  18. 2026The Impact of Likelihood Tempering on the Limiting Predictive Moments of Variational Bayesian Linear Neural Networkscited by 0