mlsanityData to 14 Oct 2026

On Convergence and Generalization of Dropout Training

arXiv 2010.12711 ↗First posted 23 Oct 2020cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

We study dropout in two-layer neural networks with rectified linear unit (ReLU) activations. Under mild overparametrization and assuming that the limiting kernel can separate the data distribution with a positive margin, we show that dropout training with logistic loss achieves ε-suboptimality in test error in O(1/ε) iterations.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 3; 2023: 3; 2024: 2; 2025: 3; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformerscited by 59
  2. 2024Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitatecited by 47
  3. 2022Implicit regularization of dropoutcited by 25
  4. 2021On the Convergence of Shallow Neural Network Training with Randomly Masked Neuronscited by 11
  5. 2023Stochastic Modified Equations and Dynamics of Dropout Algorithmcited by 11
  6. 2021Structured Dropout Variational Inference for Bayesian Neural Networkscited by 7
  7. 2024Singular-limit analysis of gradient descent with noise injectioncited by 7
  8. 2021The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularizationcited by 6
  9. 2022Weight Expansion: A New Perspective on Dropout and Generalizationcited by 6
  10. 2021What training reveals about neural network complexitycited by 5
  11. 2025Phase Diagram of Dropout for Two-Layer Neural Networks in the Mean-Field Regimecited by 3
  12. 2021Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognitioncited by 2
7 more
  1. 2021Generalization of GANs and overparameterized models under Lipschitz continuitycited by 2
  2. 2021Dropout in Training Neural Networks: Flatness of Solution and Noise Structurecited by 2
  3. 2023Robust Collaborative Inference with Vertically Split Data Over Dynamic Device Environmentscited by 2
  4. 2025Dropout Neural Network Training Viewed from a Percolation Perspectivecited by 2
  5. 2022Demystifying the Global Convergence Puzzle of Learning Over-parameterized ReLU Nets in Very High Dimensionscited by 0
  6. 2025Convergence, design and training of continuous-time dropout as a random batch methodcited by 0
  7. 2026Dropout and Random Gradient Masking Are Asymptotically Equivalent in Large ResNetscited by 0