# Survey of Dropout Methods for Deep Neural Networks

arXiv: https://arxiv.org/abs/1904.13310 · HTML: https://mlsanity.com/paper/1904.13310/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2019.

## Built on by (most cited first, 30 of 44)

- [R-Drop: Regularized Dropout for Neural Networks](https://mlsanity.com/paper/2106.14448/) (2021)
- [Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks](https://mlsanity.com/paper/2201.03299/) (2022)
- [Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate](https://mlsanity.com/paper/2402.02769/) (2024)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [EventDrop: data augmentation for event-based learning](https://mlsanity.com/paper/2106.05836/) (2021)
- [Dropout Reduces Underfitting](https://mlsanity.com/paper/2303.01500/) (2023)
- [Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimation](https://mlsanity.com/paper/2402.00728/) (2024)
- [On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons](https://mlsanity.com/paper/2112.02668/) (2021)
- [Learning Robust Real-Time Cultural Transmission without Human Data](https://mlsanity.com/paper/2203.00715/) (2022)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [Combining data assimilation and machine learning to estimate parameters of a convective-scale model](https://mlsanity.com/paper/2109.02953/) (2021)
- [A Survey on Dropout Methods and Experimental Verification in Recommendation](https://mlsanity.com/paper/2204.02027/) (2022)
- [Training neural networks with structured noise improves classification and generalization](https://mlsanity.com/paper/2302.13417/) (2023)
- [Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies](https://mlsanity.com/paper/2502.19948/) (2025)
- [The challenge of reproducible ML: an empirical study on the impact of bugs](https://mlsanity.com/paper/2109.03991/) (2021)
- [AA-Forecast: Anomaly-Aware Forecast for Extreme Events](https://mlsanity.com/paper/2208.09933/) (2022)
- [Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling](https://mlsanity.com/paper/2303.15973/) (2023)
- [Information Geometry of Dropout Training](https://mlsanity.com/paper/2206.10936/) (2022)
- [Model-based feature selection for neural networks: A mixed-integer programming approach](https://mlsanity.com/paper/2302.10344/) (2023)
- [Robust Brain MRI Image Classification with SIBOW-SVM](https://mlsanity.com/paper/2311.08908/) (2023)
- [Regularization can make diffusion models more efficient](https://mlsanity.com/paper/2502.09151/) (2025)
- [Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission](https://mlsanity.com/paper/2507.00082/) (2025)
- [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025)
- [HYPPO: A Surrogate-Based Multi-Level Parallelism Tool for Hyperparameter Optimization](https://mlsanity.com/paper/2110.01698/) (2021)
- [Jalisco's multiclass land cover analysis and classification using a novel lightweight convnet with real-world multispectral and relief data](https://mlsanity.com/paper/2201.10985/) (2022)
- [Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding](https://mlsanity.com/paper/2301.03765/) (2023)
- [Deep Augmentation: Dropout as Augmentation for Self-Supervised Learning](https://mlsanity.com/paper/2303.14537/) (2023)
- [Enhancing Split Computing and Early Exit Applications through Predefined Sparsity](https://mlsanity.com/paper/2407.11763/) (2024)
- [Deep Learning on Hester Davis Scores for Inpatient Fall Prediction](https://mlsanity.com/paper/2501.06432/) (2025)
- [Robust Face Super-Resolution and Recognition Through Multi-Feature Aggregation in Diffusion Models](https://mlsanity.com/paper/2607.05702/) (2026)

## Abstract (arXiv)

Dropout methods are a family of stochastic techniques used in neural network training or inference that have generated significant research interest and are widely used in practice. They have been successfully applied in neural network regularization, model compression, and in measuring the uncertainty of neural network outputs. While original formulated for dense neural network layers, recent advances have made dropout methods also applicable to convolutional and recurrent neural network layers. This paper summarizes the history of dropout methods, their various applications, and current areas of research interest. Important proposed methods are described in additional detail.