# Low Rank Regularization: A Review

arXiv: https://arxiv.org/abs/1808.04521 · HTML: https://mlsanity.com/paper/1808.04521/

## Key facts

- Cited by 15 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 14 Aug 2018.

## Built on by (most cited first, 15 of 15)

- [Weight decay induces low-rank attention layers](https://mlsanity.com/paper/2410.23819/) (2024)
- [Learning Sparse and Low-Rank Priors for Image Recovery via Iterative Reweighted Least Squares Minimization](https://mlsanity.com/paper/2304.10536/) (2023)
- [RHINO: Regularizing the Hash-based Implicit Neural Representation](https://mlsanity.com/paper/2309.12642/) (2023)
- [Probe-Free Low-Rank Activation Intervention](https://mlsanity.com/paper/2502.04043/) (2025)
- [Nuclear Norm Maximization Based Curiosity-Driven Learning](https://mlsanity.com/paper/2205.10484/) (2022)
- [Flat Minima in Linear Estimation and an Extended Gauss Markov Theorem](https://mlsanity.com/paper/2311.11093/) (2023)
- [Efficient Differentiable Approximation of Generalized Low-rank Regularization](https://mlsanity.com/paper/2505.15407/) (2025)
- [A Probabilistic Basis for Low-Rank Matrix Learning](https://mlsanity.com/paper/2510.05447/) (2025)
- [Self-Supervised Exploration via Temporal Inconsistency in Reinforcement Learning](https://mlsanity.com/paper/2208.11361/) (2022)
- [Reinforcement Learning for Resilient Power Grids](https://mlsanity.com/paper/2212.04069/) (2022)
- [Tensor Completion via Integer Optimization](https://mlsanity.com/paper/2402.05141/) (2024)
- [AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning](https://mlsanity.com/paper/2405.01067/) (2024)
- [Efficient Low-rank Identification via Accelerated Iteratively Reweighted Nuclear Norm Minimization](https://mlsanity.com/paper/2406.15713/) (2024)
- [Tailed Low-Rank Matrix Factorization for Similarity Matrix Completion](https://mlsanity.com/paper/2409.19550/) (2024)
- [PE-CSNet: An equivariant network architecture with learnable patch-based sparse representation](https://mlsanity.com/paper/2608.14708/) (2026)

## Abstract (arXiv)

Low rank regularization, in essence, involves introducing a low rank or approximately low rank assumption for matrix we aim to learn, which has achieved great success in many fields including machine learning, data mining and computer version. Over the last decade, much progress has been made in theories and practical applications. Nevertheless, the intersection between them is very slight. In order to construct a bridge between practical applications and theoretical research, in this paper we provide a comprehensive survey for low rank regularization. We first review several traditional machine learning models using low rank regularization, and then show their (or their variants) applications in solving practical issues, such as non-rigid structure from motion and image denoising. Subsequently, we summarize the regularizers and optimization methods that achieve great success in traditional machine learning tasks but are rarely seen in solving practical issues. Finally, we provide a discussion and comparison for some representative regularizers including convex and non-convex relaxations. Extensive experimental results demonstrate that non-convex regularizers can provide a large advantage over the nuclear norm, the regularizer widely used in solving practical issues.