# A Levenberg-Marquardt Method for Nonsmooth Regularized Least Squares

arXiv: https://arxiv.org/abs/2301.02347 · HTML: https://mlsanity.com/paper/2301.02347/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2023.

## Built on by (most cited first, 4 of 4)

- [An L-BFGS-B approach for linear and nonlinear system identification under ℓ1 and group-Lasso regularization](https://mlsanity.com/paper/2403.03827/) (2024)
- [Data-Efficient Kernel Methods for Learning Differential Equations and Their Solution Operators: Algorithms and Error Analysis](https://mlsanity.com/paper/2503.01036/) (2025)
- [A Proximal Modified Quasi-Newton Method for Nonsmooth Regularized Optimization](https://mlsanity.com/paper/2409.19428/) (2024)
- [A joint optimization approach to identifying sparse dynamics using least squares kernel collocation](https://mlsanity.com/paper/2511.18555/) (2025)

## Abstract (arXiv)

We develop a Levenberg-Marquardt method for minimizing the sum of a smooth nonlinear least-squar es term f(x) = tfrac12 |F(x)|22 and a nonsmooth term h. Both f and h may be nonconvex. Steps are computed by minimizing the sum of a regularized linear least-squares model and a model of h using a first-order method such as the proximal gradient method. We establish global convergence to a first-order stationary point of both a trust-region and a regularization variant of the Levenberg-Marquardt method under the assumptions that F and its Jacobian are Lipschitz continuous and h is proper and lower semi-continuous. In the worst case, both methods perform O(ε-2) iterations to bring a measure of stationarity below ε ∈ (0, 1). We report numerical results on three examples: a group-lasso basis-pursuit denoise example, a nonlinear support vector machine, and parameter estimation in neuron firing. For those examples to be implementable, we describe in detail how to evaluate proximal operators for separable h and for the group lasso with trust-region constraint. In all cases, the Levenberg-Marquardt methods perform fewer outer iterations than a proximal-gradient method with adaptive step length and a quasi-Newton trust-region method, neither of which exploit the least-squares structure of the problem. Our results also highlight the need for more sophisticated subproblem solvers than simple first-order methods.