# Continuous Dropout

arXiv: https://arxiv.org/abs/1911.12675 · HTML: https://mlsanity.com/paper/1911.12675/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2019.

## Built on by (most cited first, 7 of 7)

- [Bayesian Learning for Neural Networks: an algorithmic survey](https://mlsanity.com/paper/2211.11865/) (2022)
- [Self-attention fusion for audiovisual emotion recognition with incomplete data](https://mlsanity.com/paper/2201.11095/) (2022)
- [Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data](https://mlsanity.com/paper/2205.08358/) (2022)
- [Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning](https://mlsanity.com/paper/2503.21419/) (2025)
- [Towards Efficient and Data Agnostic Image Classification Training Pipeline for Embedded Systems](https://mlsanity.com/paper/2108.07049/) (2021)
- [A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing](https://mlsanity.com/paper/2607.16183/) (2026)
- [LIGAR: Lightweight General-purpose Action Recognition](https://mlsanity.com/paper/2108.13153/) (2021)

## Abstract (arXiv)

Dropout has been proven to be an effective algorithm for training robust deep networks because of its ability to prevent overfitting by avoiding the co-adaptation of feature detectors. Current explanations of dropout include bagging, naive Bayes, regularization, and sex in evolution. According to the activation patterns of neurons in the human brain, when faced with different situations, the firing rates of neurons are random and continuous, not binary as current dropout does. Inspired by this phenomenon, we extend the traditional binary dropout to continuous dropout. On the one hand, continuous dropout is considerably closer to the activation characteristics of neurons in the human brain than traditional binary dropout. On the other hand, we demonstrate that continuous dropout has the property of avoiding the co-adaptation of feature detectors, which suggests that we can extract more independent feature detectors for model averaging in the test stage. We introduce the proposed continuous dropout to a feedforward neural network and comprehensively compare it with binary dropout, adaptive dropout, and DropConnect on MNIST, CIFAR-10, SVHN, NORB, and ILSVRC-12. Thorough experiments demonstrate that our method performs better in preventing the co-adaptation of feature detectors and improves test performance. The code is available at: https://github.com/jasonustc/caffe-multigpu/tree/dropout.