# Quasi-Newton Methods: Superlinear Convergence Without Line Searches for Self-Concordant Functions

arXiv: https://arxiv.org/abs/1612.06965 · HTML: https://mlsanity.com/paper/1612.06965/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2016.

## Built on by (most cited first, 7 of 7)

- [Optimal Diagonal Preconditioning](https://mlsanity.com/paper/2209.00809/) (2022)
- [mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimization](https://mlsanity.com/paper/2307.13744/) (2023)
- [TabNSA: Native Sparse Attention for Efficient Tabular Data Learning](https://mlsanity.com/paper/2503.09850/) (2025)
- [Layer-wise Adaptive Step-Sizes for Stochastic First-Order Methods for Deep Learning](https://mlsanity.com/paper/2305.13664/) (2023)
- [Limited-Memory Greedy Quasi-Newton Method with Non-asymptotic Superlinear Convergence Rate](https://mlsanity.com/paper/2306.15444/) (2023)
- [Comparing BFGS and OGR for Second-Order Optimization](https://mlsanity.com/paper/2512.06969/) (2025)
- [From Non-Convex Self-Concordant Regularization to Scalable Quasi-Newton Training of PINNs](https://mlsanity.com/paper/2608.04206/) (2026)

## Abstract (arXiv)

We consider the use of a curvature-adaptive step size in gradient-based iterative methods, including quasi-Newton methods, for minimizing self-concordant functions, extending an approach first proposed for Newton's method by Nesterov. This step size has a simple expression that can be computed analytically; hence, line searches are not needed. We show that using this step size in the BFGS method (and quasi-Newton methods in the Broyden convex class other than the DFP method) results in superlinear convergence for strongly convex self-concordant functions. We present numerical experiments comparing gradient descent and BFGS methods using the curvature-adaptive step size to traditional methods on deterministic logistic regression problems, and to versions of stochastic gradient descent on stochastic optimization problems.