# A Framework for Variational Inference of Lightweight Bayesian Neural Networks with Heteroscedastic Uncertainties

arXiv: https://arxiv.org/abs/2402.14532 · HTML: https://mlsanity.com/paper/2402.14532/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 7 of 7)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Getting a CLUE: A Method for Explaining Uncertainty Estimates](https://mlsanity.com/paper/2006.06848/) (2020)
- [Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagation](https://mlsanity.com/paper/1908.00598/) (2019)
- [A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference](https://mlsanity.com/paper/1901.02731/) (2019)

## Built on by (most cited first, 1 of 1)

- [Few-sample Variational Inference of Bayesian Neural Networks with Arbitrary Nonlinearities](https://mlsanity.com/paper/2405.02063/) (2024)

## Abstract (arXiv)

Obtaining heteroscedastic predictive uncertainties from a Bayesian Neural Network (BNN) is vital to many applications. Often, heteroscedastic aleatoric uncertainties are learned as outputs of the BNN in addition to the predictive means, however doing so may necessitate adding more learnable parameters to the network. In this work, we demonstrate that both the heteroscedastic aleatoric and epistemic variance can be embedded into the variances of learned BNN parameters, improving predictive performance for lightweight networks. By complementing this approach with a moment propagation approach to inference, we introduce a relatively simple framework for sampling-free variational inference suitable for lightweight BNNs.