# Generalized Dropout

arXiv: https://arxiv.org/abs/1611.06791 · HTML: https://mlsanity.com/paper/1611.06791/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2016.

## Built on by (most cited first, 13 of 13)

- [Dual Lottery Ticket Hypothesis](https://mlsanity.com/paper/2203.04248/) (2022)
- [Most Activation Functions Can Win the Lottery Without Excessive Depth](https://mlsanity.com/paper/2205.02321/) (2022)
- [Winning the Lottery Ahead of Time: Efficient Early Network Pruning](https://mlsanity.com/paper/2206.10451/) (2022)
- [Plant 'n' Seek: Can You Find the Winning Ticket?](https://mlsanity.com/paper/2111.11153/) (2021)
- [Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning](https://mlsanity.com/paper/2406.01820/) (2024)
- [Is Oracle Pruning the True Oracle?](https://mlsanity.com/paper/2412.00143/) (2024)
- [Lottery Tickets with Nonzero Biases](https://mlsanity.com/paper/2110.11150/) (2021)
- [Robust Learning of Parsimonious Deep Neural Networks](https://mlsanity.com/paper/2205.04650/) (2022)
- [LOFT: Finding Lottery Tickets through Filter-wise Training](https://mlsanity.com/paper/2210.16169/) (2022)
- [Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning](https://mlsanity.com/paper/2503.21419/) (2025)
- [CODA: Constructivism Learning for Instance-Dependent Dropout Architecture Construction](https://mlsanity.com/paper/2106.08444/) (2021)
- [Adaptive Dropout for Pruning Conformers](https://mlsanity.com/paper/2412.04836/) (2024)
- [Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior](https://mlsanity.com/paper/2505.18280/) (2025)

## Abstract (arXiv)

Deep Neural Networks often require good regularizers to generalize well. Dropout is one such regularizer that is widely used among Deep Learning practitioners. Recent work has shown that Dropout can also be viewed as performing Approximate Bayesian Inference over the network parameters. In this work, we generalize this notion and introduce a rich family of regularizers which we call Generalized Dropout. One set of methods in this family, called Dropout++, is a version of Dropout with trainable parameters. Classical Dropout emerges as a special case of this method. Another member of this family selects the width of neural network layers. Experiments show that these methods help in improving generalization performance over Dropout.