mlsanityData to 7 Oct 2026

An Empirical Analysis of the Advantages of Finite- v.s. Infinite-Width Bayesian Neural Networks

arXiv 2211.09184 ↗First posted 16 Nov 2022stat.ML cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Comparing Bayesian neural networks (BNNs) with different widths is challenging because, as the width increases, multiple model properties change simultaneously, and, inference in the finite-width case is intractable. In this work, we empirically compare finite- and infinite-width BNNs, and provide quantitative and qualitative explanations for their performance difference. We find that when the model is mis-specified, increasing width can hurt BNN performance. In these cases, we provide evidence that finite-width BNNs generalize better partially due to the properties of their frequency spectrum that allows them to adapt under model mismatch.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 16 Nov 2022.

Builds on

17 of 26 references.

References · 17

  1. 2018On the Spectral Bias of Neural Networkscited by 1,321
  2. 2017Deep Neural Networks as Gaussian Processescited by 617
  3. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  4. 2019Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyrocited by 187
  5. 2019Towards Understanding the Spectral Bias of Deep Learningcited by 173
  6. 2020When Do Neural Networks Outperform Kernel Methods?cited by 150
  7. 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 83
  8. 2020Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHScited by 74
  9. 2019A Fine-Grained Spectral Perspective on Neural Networkscited by 68
  10. 2021Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeedcited by 67
  11. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 56
  12. 2014Student-t Processes as Alternatives to Gaussian Processescited by 45
5 more
  1. 2019Why bigger is not always better: on finite and infinite neural networkscited by 41
  2. 2021Precise characterization of the prior predictive distribution of deep ReLU networkscited by 28
  3. 2022Wide Mean-Field Bayesian Neural Networks Ignore the Datacited by 22
  4. 2021The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspectivecited by 21
  5. 2020Neural Network Gaussian Process Considering Input Uncertainty for Composite Structures Assemblycited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025A ZeNN architecture to avoid the Gaussian trapcited by 0