# A Bayesian Neural Network based on Dropout Regulation

arXiv: https://arxiv.org/abs/2102.01968 · HTML: https://mlsanity.com/paper/2102.01968/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Concrete Dropout](https://mlsanity.com/paper/1705.07832/) (2017)
- [Evaluating Bayesian Deep Learning Methods for Semantic Segmentation](https://mlsanity.com/paper/1811.12709/) (2018)
- [Learnable Bernoulli Dropout for Bayesian Deep Learning](https://mlsanity.com/paper/2002.05155/) (2020)

## Built on by (most cited first, 3 of 3)

- [Controlled Dropout for Uncertainty Estimation](https://mlsanity.com/paper/2205.03109/) (2022)
- [Uncertainty-Estimation with Normalized Logits for Out-of-Distribution Detection](https://mlsanity.com/paper/2302.07608/) (2023)
- [Improving Training and Inference of Face Recognition Models via Random Temperature Scaling](https://mlsanity.com/paper/2212.01015/) (2022)

## Abstract (arXiv)

Bayesian Neural Networks (BNN) have recently emerged in the Deep Learning world for dealing with uncertainty estimation in classification tasks, and are used in many application domains such as astrophysics, autonomous driving...BNN assume a prior over the weights of a neural network instead of point estimates, enabling in this way the estimation of both aleatoric and epistemic uncertainty of the model prediction.Moreover, a particular type of BNN, namely MC Dropout, assumes a Bernoulli distribution on the weights by using Dropout.Several attempts to optimize the dropout rate exist, e.g. using a variational approach.In this paper, we present a new method called "Dropout Regulation" (DR), which consists of automatically adjusting the dropout rate during training using a controller as used in automation.DR allows for a precise estimation of the uncertainty which is comparable to the state-of-the-art while remaining simple to implement.