mlsanityData to 14 Oct 2026

Greedy Quasi-Newton Methods with Explicit Superlinear Convergence

arXiv 2002.00657 ↗First posted 3 Feb 2020math.OCcited by 21

Not on the map. Why?

Abstract (arXiv)

In this paper, we study greedy variants of quasi-Newton methods. They are based on the updating formulas from a certain subclass of the Broyden family. In particular, this subclass includes the well-known DFP, BFGS and SR1 updates. However, in contrast to the classical quasi-Newton methods, which use the difference of successive iterates for updating the Hessian approximations, our methods apply basis vectors, greedily selected so as to maximize a certain measure of progress. For greedy quasi-Newton methods, we establish an explicit non-asymptotic bound on their rate of local superlinear convergence, which contains a contraction factor, depending on the square of the iteration counter. We also show that these methods produce Hessian approximations whose deviation from the exact Hessians linearly convergences to zero.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 2; 2023: 8; 2024: 5; 2025: 3; 2026: 1.

21 papers.

Sort:

  1. 2021Regularized Newton Method with Global O(1/k2) Convergencecited by 22
  2. 2022Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradientscited by 17
  3. 2023Provably Convergent Plug-and-Play Quasi-Newton Methodscited by 15
  4. 2022Hessian Averaging in Stochastic Newton Methods Achieves Superlinear Convergencecited by 14
  5. 2023Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergencecited by 11
  6. 2021Quasi-Newton Methods for Saddle Point Problems and Beyondcited by 10
  7. 2023Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimizationcited by 4
  8. 2023mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimizationcited by 4
  9. 2023Sketch-and-Project Meets Newton Method: Global mathcal O(k-2) Convergence with Low-Rank Updatescited by 3
  10. 2024Multiple Greedy Quasi-Newton Methods for Saddle Point Problemscited by 3
  11. 2025Gradient Methods with Online Scaling Part I. Theoretical Foundationscited by 3
  12. 2024Incremental Quasi-Newton Methods with Faster Superlinear Convergence Ratescited by 2
9 more
  1. 2024From Learning to Optimize to Learning Optimization Algorithmscited by 2
  2. 2024Electroencephalogram Emotion Recognition via AUC Maximizationcited by 2
  3. 2025qNBO: quasi-Newton Meets Bilevel Optimizationcited by 2
  4. 2023Sharpened Lazy Incremental Quasi-Newton Methodcited by 1
  5. 2023Self-concordant smoothing in proximal quasi-Newton algorithms for large-scale convex composite optimizationcited by 1
  6. 2023Limited-Memory Greedy Quasi-Newton Method with Non-asymptotic Superlinear Convergence Ratecited by 0
  7. 2024A Learn-to-Optimize Approach for Coordinate-Wise Step Sizes for Quasi-Newton Methodscited by 0
  8. 2025Simple Stepsize for Quasi-Newton Methods with Global Convergence Guaranteescited by 0
  9. 2026A New Kernel Regularity Condition for Distributed Mirror Descent: Broader Coverage and Simpler Analysiscited by 0