# On implicit regularization: Morse functions and applications to matrix factorization

arXiv: https://arxiv.org/abs/2001.04264 · HTML: https://mlsanity.com/paper/2001.04264/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2020.

## Built on by (most cited first, 11 of 11)

- [Trained Transformers Learn Linear Models In-Context](https://mlsanity.com/paper/2306.09927/) (2023)
- [On the Implicit Bias in Deep-Learning Algorithms](https://mlsanity.com/paper/2208.12591/) (2022)
- [Implicit Regularization Towards Rank Minimization in ReLU Networks](https://mlsanity.com/paper/2201.12760/) (2022)
- [Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing](https://mlsanity.com/paper/2301.11500/) (2023)
- [On Margin Maximization in Linear and ReLU Networks](https://mlsanity.com/paper/2110.02732/) (2021)
- [The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks](https://mlsanity.com/paper/2306.17499/) (2023)
- [The Inductive Bias of Flatness Regularization for Deep Matrix Factorization](https://mlsanity.com/paper/2306.13239/) (2023)
- [Neural McKean-Vlasov Processes: Distributional Dependence in Diffusion Processes](https://mlsanity.com/paper/2404.09402/) (2024)
- [Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion](https://mlsanity.com/paper/2405.13721/) (2024)
- [Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning](https://mlsanity.com/paper/2408.13767/) (2024)
- [Implicit Bias in Matrix Factorization and its Explicit Realization in a New Architecture](https://mlsanity.com/paper/2501.16322/) (2025)

## Abstract (arXiv)

In this paper, we revisit implicit regularization from the ground up using notions from dynamical systems and invariant subspaces of Morse functions. The key contributions are a new criterion for implicit regularization---a leading contender to explain the generalization power of deep models such as neural networks---and a general blueprint to study it. We apply these techniques to settle a conjecture on implicit regularization in matrix factorization.