mlsanityData to 7 Oct 2026

A Linearly-Convergent Stochastic L-BFGS Algorithm

arXiv 1508.02087 ↗First posted 9 Aug 2015math.OC cs.LG cs.NAcited by 39

Not on the map. Why?

Abstract (arXiv)

We propose a new stochastic L-BFGS algorithm and prove a linear convergence rate for strongly convex and smooth functions. Our algorithm draws heavily from a recent stochastic variant of L-BFGS proposed in Byrd et al. (2014) as well as a recent approach to variance reduction for stochastic gradient descent from Johnson and Zhang (2013). We demonstrate experimentally that our algorithm performs well on large-scale convex and non-convex optimization problems, exhibiting linear convergence and rapidly solving the optimization problems to high levels of precision. Furthermore, we show that our algorithm performs well for a wide-range of step sizes, often differing by several orders of magnitude.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Aug 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 10; 2023: 8; 2024: 10; 2025: 5; 2026: 2.

39 papers.

Sort:

By area:

  1. 2022An Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methodscited by 63
  2. 2022SP2: A Second Order Stochastic Polyak Methodcited by 13
  3. 2022Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batchescited by 9
  4. 2022SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimatescited by 8
  5. 2023Curvature-Aware Training for Coordinate Networkscited by 8
  6. 2021Nys-Newton: Nystr"om-Approximated Curvature for Stochastic Optimizationcited by 4
  7. 2024SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrixcited by 4
  8. 2025SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learningcited by 4
  9. 2022FedSSO: A Federated Server-Side Second-Order Optimization Algorithmcited by 3
  10. 2023Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimizationcited by 3
  11. 2023MLPGradientFlow: going with the flow of multilayer perceptrons (and finding minima fast and accurately)cited by 3
  12. 2023mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimizationcited by 3
27 more
  1. 2023Exponential Quantum Communication Advantage in Distributed Inference and Learningcited by 3
  2. 2024Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradientscited by 3
  3. 2024ONER: Online Experience Replay for Incremental Anomaly Detectioncited by 3
  4. 2021Quasi-Newton Quasi-Monte Carlo for variational Bayescited by 2
  5. 2021Exploiting Local Convergence of Quasi-Newton Methods Globally: Adaptive Sample Size Approachcited by 2
  6. 2022Stochastic Steffensen methodcited by 2
  7. 2024Incremental Quasi-Newton Methods with Faster Superlinear Convergence Ratescited by 2
  8. 2025TabNSA: Native Sparse Attention for Efficient Tabular Data Learningcited by 2
  9. 2021Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learningcited by 1
  10. 2022On the efficiency of Stochastic Quasi-Newton Methods for Deep Learningcited by 1
  11. 2022Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descentcited by 1
  12. 2023Sharpened Lazy Incremental Quasi-Newton Methodcited by 1
  13. 2024A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothnesscited by 1
  14. 2024A neural network approach for solving the Monge-Amp`ere equation with transport boundary conditioncited by 1
  15. 2024An Evaluation Framework for Product Images Background Inpainting based on Human Feedback and Product Consistencycited by 1
  16. 2025DexAvatar: 3D Sign Language Reconstruction with Hand and Body Pose Priorscited by 1
  17. 2026Neural-network methods for two-dimensional finite-source reflector designcited by 1
  18. 2022SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimizationcited by 0
  19. 2022A Stochastic Variance Reduced Gradient using Barzilai-Borwein Techniques as Second Order Informationcited by 0
  20. 2023Online Learning Under A Separable Stochastic Approximation Frameworkcited by 0
  21. 2023Limited-Memory Greedy Quasi-Newton Method with Non-asymptotic Superlinear Convergence Ratecited by 0
  22. 2024Physics-Informed Neural Network Based Digital Image Correlation Methodcited by 0
  23. 2024Kolmogorov-Arnold Neural Networks for High-Entropy Alloys Designcited by 0
  24. 2024A Bayesian explanation of machine learning models based on modes and functional ANOVAcited by 0
  25. 2025Mo' Memory, Mo' Problems: Stream-Native Machine Unlearningcited by 0
  26. 2025Physics-Informed Neural Network Frameworks for the Analysis of Engineering and Biological Dynamical Systems Governed by Ordinary Differential Equationscited by 0
  27. 2026Direct Optimization of a 3D Finite-Source Reflector via Neural-Network Parameterizationcited by 0