# LSOS: Line-search Second-Order Stochastic optimization methods for nonconvex finite sums

arXiv: https://arxiv.org/abs/2007.15966 · HTML: https://mlsanity.com/paper/2007.15966/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jul 2020.

## Built on by (most cited first, 4 of 4)

- [An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians](https://mlsanity.com/paper/2102.05320/) (2021)
- [Inequality Constrained Stochastic Nonlinear Optimization via Active-Set Sequential Quadratic Programming](https://mlsanity.com/paper/2109.11502/) (2021)
- [Numerical Solution of Stiff ODEs with Physics-Informed RPNNs](https://mlsanity.com/paper/2108.01584/) (2021)
- [FUSE: First-Order and Second-Order Unified SynthEsis in Stochastic Optimization](https://mlsanity.com/paper/2503.04204/) (2025)

## Abstract (arXiv)

We develop a line-search second-order algorithmic framework for minimizing finite sums. We do not make any convexity assumptions, but require the terms of the sum to be continuously differentiable and have Lipschitz-continuous gradients. The methods fitting into this framework combine line searches and suitably decaying step lengths. A key issue is a two-step sampling at each iteration, which allows us to control the error present in the line-search procedure. Stationarity of limit points is proved in the almost-sure sense, while almost-sure convergence of the sequence of approximations to the solution holds with the additional hypothesis that the functions are strongly convex. Numerical experiments, including comparisons with state-of-the art stochastic optimization methods, show the efficiency of our approach.