mlsanityData to 7 Oct 2026

Function-Space MCMC for Bayesian Wide Neural Networks

arXiv 2408.14325 ↗First posted 26 Aug 2024cs.LG stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

Bayesian Neural Networks represent a fascinating confluence of deep learning and probabilistic reasoning, offering a compelling framework for understanding uncertainty in complex predictive models. In this paper, we investigate the use of the preconditioned Crank-Nicolson algorithm and its Langevin version to sample from a reparametrised posterior distribution of the neural network's weights, as the widths grow larger. In addition to being robust in the infinite-dimensional setting, we prove that the acceptance probabilities of the proposed algorithms approach 1 as the width of the network increases, independently of any stepsize tuning. Moreover, we examine and compare how the mixing speeds of the underdamped Langevin Monte Carlo, the preconditioned Crank-Nicolson and the preconditioned Crank-Nicolson Langevin samplers are influenced by changes in the network width in some real-world cases. Our findings suggest that, in wide Bayesian Neural Networks configurations, the preconditioned Crank-Nicolson algorithm allows for a scalable and more efficient sampling of the reparametrised posterior distribution, as also evidenced by a higher effective sample size and improved diagnostic results compared with the other analysed algorithms.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 26 Aug 2024.

Builds on

20 of 30 references.

References · 20

  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2017Deep Neural Networks as Gaussian Processescited by 617
  4. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  5. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  6. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  7. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  8. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 164
  9. 2019Practical Deep Learning with Bayesian Principlescited by 158
  10. 2019Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processescited by 148
  11. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 137
  12. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
8 more
  1. 2017TensorFlow Distributionscited by 105
  2. 2020Bayesian Deep Learning via Subnetwork Inferencecited by 85
  3. 2019Why bigger is not always better: on finite and infinite neural networkscited by 41
  4. 2020Exact posterior distributions of wide Bayesian neural networkscited by 29
  5. 2021A theory of representation learning gives a deep generalisation of kernel methodscited by 15
  6. 2021Large-width functional asymptotics for deep Gaussian neural networkscited by 14
  7. 2022Wide Bayesian neural networks have a simple weight posterior: theory and accelerated samplingcited by 9
  8. 2020Trace-class Gaussian priors for Bayesian learning of neural networks with MCMCcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Posterior Bayesian Neural Networks with Dependent Weightscited by 0