# Quasi-Newton Updating for Large-Scale Distributed Learning

arXiv: https://arxiv.org/abs/2306.04111 · HTML: https://mlsanity.com/paper/2306.04111/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Communication Efficient Distributed Optimization using an Approximate Newton-type Method](https://mlsanity.com/paper/1312.7853/) (2013)
- [GIANT: Globally Improved Approximate Newton Method for Distributed Optimization](https://mlsanity.com/paper/1709.03528/) (2017)
- [Accelerated Distributed Nesterov Gradient Descent](https://mlsanity.com/paper/1705.07176/) (2017)
- [DINGO: Distributed Newton-Type Method for Gradient-Norm Optimization](https://mlsanity.com/paper/1901.05134/) (2019)
- [A Distributed One-Step Estimator](https://mlsanity.com/paper/1511.01443/) (2015)
- [IQN: An Incremental Quasi-Newton Method with Local Superlinear Convergence Rate](https://mlsanity.com/paper/1702.00709/) (2017)
- [Least Squares Approximation for a Distributed System](https://mlsanity.com/paper/1908.04904/) (2019)
- [A Distributed and Integrated Method of Moments for High-Dimensional Correlated Data Analysis](https://mlsanity.com/paper/1910.02986/) (2019)
- [DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate](https://mlsanity.com/paper/1906.00506/) (2019)
- [Decentralized Quasi-Newton Methods](https://mlsanity.com/paper/1605.00933/) (2016)
- [A Distributed Quasi-Newton Algorithm for Empirical Risk Minimization with Nonsmooth Regularization](https://mlsanity.com/paper/1803.01370/) (2018)
- [Distributed Feature Screening via Componentwise Debiasing](https://mlsanity.com/paper/1903.03810/) (2019)
- [Efficient Estimation for Generalized Linear Models on a Distributed System with Nonrandomly Distributed Data](https://mlsanity.com/paper/2004.02414/) (2020)

## Built on by (most cited first, 2 of 2)

- [Byzantine-tolerant distributed learning of finite mixture models](https://mlsanity.com/paper/2407.13980/) (2024)
- [A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques](https://mlsanity.com/paper/2403.11163/) (2024)

## Abstract (arXiv)

Distributed computing is critically important for modern statistical analysis. Herein, we develop a distributed quasi-Newton (DQN) framework with excellent statistical, computation, and communication efficiency. In the DQN method, no Hessian matrix inversion or communication is needed. This considerably reduces the computation and communication complexity of the proposed method. Notably, related existing methods only analyze numerical convergence and require a diverging number of iterations to converge. However, we investigate the statistical properties of the DQN method and theoretically demonstrate that the resulting estimator is statistically efficient over a small number of iterations under mild conditions. Extensive numerical analyses demonstrate the finite sample performance.