# On the Acceleration of L-BFGS with Second-Order Information and Stochastic Batches

arXiv: https://arxiv.org/abs/1807.05328 · HTML: https://mlsanity.com/paper/1807.05328/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jul 2018.

## Built on by (most cited first, 1 of 1)

- [Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysis](https://mlsanity.com/paper/2110.07567/) (2021)

## Abstract (arXiv)

This paper proposes a framework of L-BFGS based on the (approximate) second-order information with stochastic batches, as a novel approach to the finite-sum minimization problems. Different from the classical L-BFGS where stochastic batches lead to instability, we use a smooth estimate for the evaluations of the gradient differences while achieving acceleration by well-scaling the initial Hessians. We provide theoretical analyses for both convex and nonconvex cases. In addition, we demonstrate that within the popular applications of least-square and cross-entropy losses, the algorithm admits a simple implementation in the distributed environment. Numerical experiments support the efficiency of our algorithms.