# Dropout Rademacher Complexity of Deep Neural Networks

arXiv: https://arxiv.org/abs/1402.3811 · HTML: https://mlsanity.com/paper/1402.3811/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2014.

## Built on by (most cited first, 14 of 14)

- [Reflash Dropout in Image Super-Resolution](https://mlsanity.com/paper/2112.12089/) (2021)
- [Controlling Directions Orthogonal to a Classifier](https://mlsanity.com/paper/2201.11259/) (2022)
- [LocalDrop: A Hybrid Regularization for Deep Neural Networks](https://mlsanity.com/paper/2103.00719/) (2021)
- [Generalization and Estimation Error Bounds for Model-based Neural Networks](https://mlsanity.com/paper/2304.09802/) (2023)
- [What training reveals about neural network complexity](https://mlsanity.com/paper/2106.04186/) (2021)
- [Benefits of Additive Noise in Composing Classes with Bounded Capacity](https://mlsanity.com/paper/2206.07199/) (2022)
- [An Anomaly Detection Method for Satellites Using Monte Carlo Dropout](https://mlsanity.com/paper/2211.14938/) (2022)
- [MESAHA-Net: Multi-Encoders based Self-Adaptive Hard Attention Network with Maximum Intensity Projections for Lung Nodule Segmentation in CT Scan](https://mlsanity.com/paper/2304.01576/) (2023)
- [Dropout Drops Double Descent](https://mlsanity.com/paper/2305.16179/) (2023)
- [Complementary Labels Learning with Augmented Classes](https://mlsanity.com/paper/2211.10701/) (2022)
- [On the Role of Noise in the Sample Complexity of Learning Recurrent Neural Networks: Exponential Gaps for Long Sequences](https://mlsanity.com/paper/2305.18423/) (2023)
- [Consistency for Large Neural Networks: Regression and Classification](https://mlsanity.com/paper/2409.14123/) (2024)
- [ESA: Example Sieve Approach for Multi-Positive and Unlabeled Learning](https://mlsanity.com/paper/2412.02240/) (2024)
- [Dropout Prompt Learning: Towards Robust and Adaptive Vision-Language Models](https://mlsanity.com/paper/2512.07234/) (2025)

## Abstract (arXiv)

Great successes of deep neural networks have been witnessed in various real applications. Many algorithmic and implementation techniques have been developed, however, theoretical understanding of many aspects of deep neural networks is far from clear. A particular interesting issue is the usefulness of dropout, which was motivated from the intuition of preventing complex co-adaptation of feature detectors. In this paper, we study the Rademacher complexity of different types of dropout, and our theoretical results disclose that for shallow neural networks (with one or none hidden layer) dropout is able to reduce the Rademacher complexity in polynomial, whereas for deep neural networks it can amazingly lead to an exponential reduction of the Rademacher complexity.