# Dropout Regularization Versus ℓ2-Penalization in the Linear Model

arXiv: https://arxiv.org/abs/2306.10529 · HTML: https://mlsanity.com/paper/2306.10529/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2023.

## Built on by (most cited first, 1 of 1)

- [Hebbian learning inspired estimation of the linear regression parameters from queries](https://mlsanity.com/paper/2311.03483/) (2023)

## Abstract (arXiv)

We investigate the statistical behavior of gradient descent iterates with dropout in the linear regression model. In particular, non-asymptotic bounds for the convergence of expectations and covariance matrices of the iterates are derived. The results shed more light on the widely cited connection between dropout and l2-regularization in the linear model. We indicate a more subtle relationship, owing to interactions between the gradient descent dynamics and the additional randomness induced by dropout. Further, we study a simplified variant of dropout which does not have a regularizing effect and converges to the least squares estimator