# Regularization of Limited Memory Quasi-Newton Methods for Large-Scale Nonconvex Minimization

arXiv: https://arxiv.org/abs/1911.04584 · HTML: https://mlsanity.com/paper/1911.04584/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 3 of 3)

- [A Regularized Limited Memory BFGS method for Large-Scale Unconstrained Optimization and its Efficient Implementations](https://mlsanity.com/paper/2101.04413/) (2021)
- [Useful Compact Representations for Data-Fitting](https://mlsanity.com/paper/2403.12206/) (2024)
- [From Non-Convex Self-Concordant Regularization to Scalable Quasi-Newton Training of PINNs](https://mlsanity.com/paper/2608.04206/) (2026)

## Abstract (arXiv)

This paper deals with regularized Newton methods, a flexible class of unconstrained optimization algorithms that is competitive with line search and trust region methods and potentially combines attractive elements of both. The particular focus is on combining regularization with limited memory quasi-Newton methods by exploiting the special structure of limited memory algorithms. Global convergence of regularization methods is shown under mild assumptions and the details of regularized limited memory quasi-Newton updates are discussed including their compact representations. Numerical results using all large-scale test problems from the CUTEst collection indicate that our regularized version of L-BFGS is competitive with state-of-the-art line search and trust-region L-BFGS algorithms and previous attempts at combining L-BFGS with regularization, while potentially outperforming some of them, especially when nonmonotonicity is involved.