# Split personalities in Bayesian Neural Networks: the case for full marginalisation

arXiv: https://arxiv.org/abs/2205.11151 · HTML: https://mlsanity.com/paper/2205.11151/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 11 of 11)

- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning](https://mlsanity.com/paper/2106.04015/) (2021)
- [The Case for Bayesian Deep Learning](https://mlsanity.com/paper/2001.10995/) (2020)
- [Learning in High Dimension Always Amounts to Extrapolation](https://mlsanity.com/paper/2110.09485/) (2021)
- [Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting](https://mlsanity.com/paper/2010.06772/) (2020)
- [What is an equivariant neural network?](https://mlsanity.com/paper/2205.07362/) (2022)
- [Compromise-free Bayesian neural networks](https://mlsanity.com/paper/2004.12211/) (2020)
- [Bayesian sparse reconstruction: a brute-force approach to astronomical imaging and machine learning](https://mlsanity.com/paper/1809.04598/) (2018)

## Built on by (most cited first, 1 of 1)

- [Robust Simulation-Based Inference in Cosmology with Bayesian Neural Networks](https://mlsanity.com/paper/2207.08435/) (2022)

## Abstract (arXiv)

The true posterior distribution of a Bayesian neural network is massively multimodal. Whilst most of these modes are functionally equivalent, we demonstrate that there remains a level of real multimodality that manifests in even the simplest neural network setups. It is only by fully marginalising over all posterior modes, using appropriate Bayesian sampling tools, that we can capture the split personalities of the network. The ability of a network trained in this manner to reason between multiple candidate solutions dramatically improves the generalisability of the model, a feature we contend is not consistently captured by alternative approaches to the training of Bayesian neural networks. We provide a concise minimal example of this, which can provide lessons and a future path forward for correctly utilising the explainability and interpretability of Bayesian neural networks.