mlsanityData to 7 Oct 2026

On the Implicit Bias of Dropout

arXiv 1806.09777 ↗First posted 26 Jun 2018cs.LG cs.AI stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Algorithmic approaches endow deep learning systems with implicit bias that helps them generalize even in over-parametrized settings. In this paper, we focus on understanding such a bias induced in learning through dropout, a popular technique to avoid overfitting in deep learning. For single hidden-layer linear neural networks, we show that dropout tends to make the norm of incoming/outgoing weight vectors of all the hidden nodes equal. In addition, we provide a complete characterization of the optimization landscape induced by dropout.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 3; 2024: 1; 2025: 4; 2026: 4.

22 papers.

Sort:

  1. 2021Label Noise SGD Provably Prefers Flat Global Minimizerscited by 121
  2. 2022The dynamics of representation learning in shallow, non-linear autoencoderscited by 27
  3. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  4. 2022Implicit regularization of dropoutcited by 24
  5. 2022The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspectivecited by 24
  6. 2023A Unified Approach to Controlling Implicit Regularization via Mirror Descentcited by 13
  7. 2021Structured Dropout Variational Inference for Bayesian Neural Networkscited by 7
  8. 2025An Overview of Low-Rank Structures in the Training and Adaptation of Large Modelscited by 7
  9. 2022Weight Expansion: A New Perspective on Dropout and Generalizationcited by 6
  10. 2024Singular-limit analysis of gradient descent with noise injectioncited by 5
  11. 2026On Surprising Effectiveness of Masking Updates in Adaptive Optimizerscited by 5
  12. 2021An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Networkcited by 3
10 more
  1. 2025Analytic theory of dropout regularizationcited by 2
  2. 2025Dropout Neural Network Training Viewed from a Percolation Perspectivecited by 2
  3. 2025Overlap-Adaptive Regularization for Conditional Average Treatment Effect Estimationcited by 1
  4. 2021Advances in Electron Microscopy with Deep Learningcited by 0
  5. 2021A Reflection on Learning from Data: Epistemology Issues and Limitationscited by 0
  6. 2022Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provablycited by 0
  7. 2023ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden Statescited by 0
  8. 2026Estimating Implicit Regularization in Deep Learningcited by 0
  9. 2026Dropout and Random Gradient Masking Are Asymptotically Equivalent in Large ResNetscited by 0
  10. 2026Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradoxcited by 0