# Dropout as a Low-Rank Regularizer for Matrix Factorization

arXiv: https://arxiv.org/abs/1710.05092 · HTML: https://mlsanity.com/paper/1710.05092/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2017.

## Built on by (most cited first, 14 of 14)

- [Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks](https://mlsanity.com/paper/2201.03299/) (2022)
- [Implicit regularization of dropout](https://mlsanity.com/paper/2207.05952/) (2022)
- [Exact Solutions of a Deep Linear Network](https://mlsanity.com/paper/2202.04777/) (2022)
- [The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective](https://mlsanity.com/paper/2210.05021/) (2022)
- [An Overview of Low-Rank Structures in the Training and Adaptation of Large Models](https://mlsanity.com/paper/2503.19859/) (2025)
- [Regularization in ResNet with Stochastic Depth](https://mlsanity.com/paper/2106.03091/) (2021)
- [Weight Expansion: A New Perspective on Dropout and Generalization](https://mlsanity.com/paper/2201.09209/) (2022)
- [Stochastic Neural Networks with Infinite Width are Deterministic](https://mlsanity.com/paper/2201.12724/) (2022)
- [Near-Linear Time Projection onto the ℓ1,∞ Ball; Application to Sparse Autoencoders](https://mlsanity.com/paper/2307.09836/) (2023)
- [Estimating Implicit Regularization in Deep Learning](https://mlsanity.com/paper/2605.05436/) (2026)
- [Multi-level projection with exponential parallel speedup; Application to sparse auto-encoders neural networks](https://mlsanity.com/paper/2405.02086/) (2024)
- [A new Linear Time Bi-level ℓ1,∞ projection ; Application to the sparsification of auto-encoders neural networks](https://mlsanity.com/paper/2407.16293/) (2024)
- [Does Weighting Improve Matrix Factorization for Recommender Systems?](https://mlsanity.com/paper/2510.10440/) (2025)
- [On the Regularization Landscape for the Linear Recommendation Models](https://mlsanity.com/paper/2609.11876/) (2026)

## Abstract (arXiv)

Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain quite elusive for this class of problems. In this paper, we present a theoretical analysis of dropout for MF, where Bernoulli random variables are used to drop columns of the factors. We demonstrate the equivalence between dropout and a fully deterministic model for MF in which the factors are regularized by the sum of the product of squared Euclidean norms of the columns. Additionally, we inspect the case of a variable sized factorization and we prove that dropout achieves the global minimum of a convex approximation problem with (squared) nuclear norm regularization. As a result, we conclude that dropout can be used as a low-rank regularizer with data dependent singular-value thresholding.