# A Bayesian encourages dropout

arXiv: https://arxiv.org/abs/1412.7003 · HTML: https://mlsanity.com/paper/1412.7003/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2014.

## Built on by (most cited first, 6 of 6)

- [Structured Dropout Variational Inference for Bayesian Neural Networks](https://mlsanity.com/paper/2102.07927/) (2021)
- [MLAE: Masked LoRA Experts for Visual Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2405.18897/) (2024)
- [An Uncertainty Estimation Framework for Probabilistic Object Detection](https://mlsanity.com/paper/2106.15007/) (2021)
- [4D flight trajectory prediction using a hybrid Deep Learning prediction method based on ADS-B technology: a case study of Hartsfield-Jackson Atlanta International Airport(ATL)](https://mlsanity.com/paper/2110.07774/) (2021)
- [Accurate Prediction and Uncertainty Estimation using Decoupled Prediction Interval Networks](https://mlsanity.com/paper/2202.09664/) (2022)
- [CODA: Constructivism Learning for Instance-Dependent Dropout Architecture Construction](https://mlsanity.com/paper/2106.08444/) (2021)

## Abstract (arXiv)

Dropout is one of the key techniques to prevent the learning from overfitting. It is explained that dropout works as a kind of modified L2 regularization. Here, we shed light on the dropout from Bayesian standpoint. Bayesian interpretation enables us to optimize the dropout rate, which is beneficial for learning of weight parameters and prediction after learning. The experiment result also encourages the optimization of the dropout.