# Curriculum Dropout

arXiv: https://arxiv.org/abs/1703.06229 · HTML: https://mlsanity.com/paper/1703.06229/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2017.

## Built on by (most cited first, 30 of 50)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Geometry Uncertainty Projection Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2107.13774/) (2021)
- [Curriculum Temperature for Knowledge Distillation](https://mlsanity.com/paper/2211.16231/) (2022)
- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones](https://mlsanity.com/paper/2211.09703/) (2022)
- [Curriculum Direct Preference Optimization for Diffusion and Consistency Models](https://mlsanity.com/paper/2405.13637/) (2024)
- [On-the-fly Modulation for Balanced Multimodal Learning](https://mlsanity.com/paper/2410.11582/) (2024)
- [Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers](https://mlsanity.com/paper/2205.14336/) (2022)
- [Dropout Reduces Underfitting](https://mlsanity.com/paper/2303.01500/) (2023)
- [DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization](https://mlsanity.com/paper/2308.10285/) (2023)
- [EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training](https://mlsanity.com/paper/2405.08768/) (2024)
- [Dynamic Masking Rate Schedules for MLM Pretraining](https://mlsanity.com/paper/2305.15096/) (2023)
- [Unsupervised Medical Image Alignment with Curriculum Learning](https://mlsanity.com/paper/2102.10438/) (2021)
- [RecRecNet: Rectangling Rectified Wide-Angle Images by Thin-Plate Spline Model and DoF-based Curriculum Learning](https://mlsanity.com/paper/2301.01661/) (2023)
- [G2NetPL: Generic Game-Theoretic Network for Partial-Label Image Classification](https://mlsanity.com/paper/2210.11469/) (2022)
- [Easy Begun is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout](https://mlsanity.com/paper/2211.15182/) (2022)
- [Multifidelity data fusion in convolutional encoder/decoder networks](https://mlsanity.com/paper/2205.05187/) (2022)
- [CRISP: Curriculum based Sequential Neural Decoders for Polar Code Family](https://mlsanity.com/paper/2210.00313/) (2022)
- [An Effective Approach for Multi-label Classification with Missing Labels](https://mlsanity.com/paper/2210.13651/) (2022)
- [GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2310.15624/) (2023)
- [History-Aware Neural Operator: Robust Data-Driven Constitutive Modeling of Path-Dependent Materials](https://mlsanity.com/paper/2506.10352/) (2025)
- [Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails](https://mlsanity.com/paper/2108.00602/) (2021)
- [A Survey on Dropout Methods and Experimental Verification in Recommendation](https://mlsanity.com/paper/2204.02027/) (2022)
- [Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning](https://mlsanity.com/paper/2503.21419/) (2025)
- [A statistical physics framework for optimal learning](https://mlsanity.com/paper/2507.07907/) (2025)
- [PLACE dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization](https://mlsanity.com/paper/2112.03676/) (2021)
- [MetaVA: Curriculum Meta-learning and Pre-fine-tuning of Deep Neural Networks for Detecting Ventricular Arrhythmias based on ECGs](https://mlsanity.com/paper/2202.12450/) (2022)
- [The Overfocusing Bias of Convolutional Neural Networks: A Saliency-Guided Regularization Approach](https://mlsanity.com/paper/2409.17370/) (2024)
- [Analytic theory of dropout regularization](https://mlsanity.com/paper/2505.07792/) (2025)
- [Argus: Federated Non-convex Bilevel Learning over 6G Space-Air-Ground Integrated Network](https://mlsanity.com/paper/2505.09106/) (2025)

## Abstract (arXiv)

Dropout is a very effective way of regularizing neural networks. Stochastically "dropping out" units with a certain probability discourages over-specific co-adaptations of feature detectors, preventing overfitting and improving network generalization. Besides, Dropout can be interpreted as an approximate model aggregation technique, where an exponential number of smaller networks are averaged in order to get a more powerful ensemble. In this paper, we show that using a fixed dropout probability during training is a suboptimal choice. We thus propose a time scheduling for the probability of retaining neurons in the network. This induces an adaptive regularization scheme that smoothly increases the difficulty of the optimization problem. This idea of "starting easy" and adaptively increasing the difficulty of the learning problem has its roots in curriculum learning and allows one to train better models. Indeed, we prove that our optimization strategy implements a very general curriculum scheme, by gradually adding noise to both the input and intermediate feature representations within the network architecture. Experiments on seven image classification datasets and different network architectures show that our method, named Curriculum Dropout, frequently yields to better generalization and, at worst, performs just as well as the standard Dropout method.