mlsanityData to 14 Oct 2026

Dropout: Explicit Forms and Capacity Control

arXiv 2003.03397 ↗First posted 6 Mar 2020cs.LG stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

We investigate the capacity control provided by dropout in various machine learning problems. First, we study dropout for matrix completion, where it induces a data-dependent regularizer that, in expectation, equals the weighted trace-norm of the product of the factors. In deep learning, we show that the data-dependent regularizer due to dropout directly controls the Rademacher complexity of the underlying class of deep neural networks. These developments enable us to give concrete generalization error bounds for the dropout algorithm in both matrix completion as well as training deep neural networks. We evaluate our theoretical findings on real-world datasets, including MovieLens, MNIST, and Fashion-MNIST.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 9; 2023: 2; 2024: 1; 2025: 7; 2026: 2.

27 papers.

Sort:

By area:

  1. 2021Meta-Learning with Fewer Tasks through Task Interpolationcited by 45
  2. 2021Noisy Recurrent Neural Networkscited by 42
  3. 2022UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixupcited by 34
  4. 2022A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspectivecited by 27
  5. 2022Implicit regularization of dropoutcited by 25
  6. 2022Exact Solutions of a Deep Linear Networkcited by 24
  7. 2021Transformers are Deep Infinite-Dimensional Non-Mercer Binary Kernel Machinescited by 22
  8. 2022Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformerscited by 22
  9. 2024Efficient Stagewise Pretraining via Progressive Subnetworkscited by 10
  10. 2025An Overview of Low-Rank Structures in the Training and Adaptation of Large Modelscited by 9
  11. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  12. 2021The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularizationcited by 6
15 more
  1. 2022Towards Size-Independent Generalization Bounds for Deep Operator Netscited by 6
  2. 2021What training reveals about neural network complexitycited by 5
  3. 2022Stochastic Neural Networks with Infinite Width are Deterministiccited by 3
  4. 2022Boosting Factorization Machines via Saliency-Guided Mixupcited by 3
  5. 2025Phase Diagram of Dropout for Two-Layer Neural Networks in the Mean-Field Regimecited by 3
  6. 2021Generalization of GANs and overparameterized models under Lipschitz continuitycited by 2
  7. 2023Reweighted Mixup for Subpopulation Shiftcited by 2
  8. 2023Dropout Drops Double Descentcited by 2
  9. 2025Analytic theory of dropout regularizationcited by 2
  10. 2022Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reductioncited by 1
  11. 2025LangDAug: Langevin Data Augmentation for Multi-Source Domain Generalization in Medical Image Segmentationcited by 1
  12. 2025Overlap-Adaptive Regularization for Conditional Average Treatment Effect Estimationcited by 1
  13. 2025The Hidden Power of Normalization Layers in Neural Networks: Exponential Capacity Controlcited by 0
  14. 2026Explicit Dropout: Deterministic Regularization for Transformer Architecturescited by 0
  15. 2026Dropout and Random Gradient Masking Are Asymptotically Equivalent in Large ResNetscited by 0