mlsanityData to 7 Oct 2026

GIANT: Globally Improved Approximate Newton Method for Distributed Optimization

arXiv 1709.03528 ↗First posted 11 Sep 2017cs.LG cs.DC math.OCcited by 43

Not on the map. Why?

Abstract (arXiv)

For distributed computing environment, we consider the empirical risk minimization problem and propose a distributed and communication-efficient Newton-type optimization method. At every iteration, each worker locally finds an Approximate NewTon (ANT) direction, which is sent to the main driver. The main driver, then, averages all the ANT directions received from workers to form a it Globally Improved ANT (GIANT) direction. GIANT is highly communication efficient and naturally exploits the trade-offs between local computations and global communications in that more local computations result in fewer overall rounds of communications. Theoretically, we show that GIANT enjoys an improved convergence rate as compared with first-order methods and existing distributed Newton-type methods. Further, and in sharp contrast with many existing distributed Newton-type methods, as well as popular first-order methods, a highly advantageous practical feature of GIANT is that it only involves one tuning parameter. We conduct large-scale experiments on a computer cluster and, empirically, demonstrate the superior performance of GIANT.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 8; 2023: 6; 2024: 8; 2025: 4; 2026: 2.

43 papers.

Sort:

  1. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  2. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  3. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  4. 2021Hessian Eigenspectra of More Realistic Nonlinear Modelscited by 33
  5. 2021Distributed Second Order Methods with Fast Rates and Compressed Communicationcited by 25
  6. 2021Newton-LESS: Sparsification without Trade-offs for the Sketched Newton Updatecited by 23
  7. 2023Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oraclescited by 16
  8. 2021LocalNewton: Reducing Communication Bottleneck for Distributed Learningcited by 15
  9. 2021Adaptive Newton Sketch: Linear-time Optimization with Quadratic Convergence and Effective Hessian Dimensionalitycited by 14
  10. 2021Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learningcited by 14
  11. 2022SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharingcited by 11
  12. 2022Over-the-Air Federated Learning via Second-Order Optimizationcited by 11
31 more
  1. 2022Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregationcited by 11
  2. 2021Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysiscited by 10
  3. 2024Federated Optimization with Doubly Regularized Drift Correctioncited by 10
  4. 2021On Second-order Optimization Methods for Federated Learningcited by 9
  5. 2021Distributed Saddle-Point Problems Under Similaritycited by 8
  6. 2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
  7. 2021A Stochastic Newton Algorithm for Distributed Convex Optimizationcited by 6
  8. 2022Federated Learning with Erroneous Communication Linkscited by 5
  9. 2022Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration and Lower Boundscited by 5
  10. 2025Distributed Quasi-Newton Method for Fair and Fast Federated Learningcited by 4
  11. 2021Guided parallelized stochastic gradient descent for delay compensationcited by 3
  12. 2022FedSSO: A Federated Server-Side Second-Order Optimization Algorithmcited by 3
  13. 2024Distributed Least Squares in Small Space via Sketching and Bias Reductioncited by 3
  14. 2024High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updatescited by 3
  15. 2021Delayed Projection Techniques for Linearly Constrained Problems: Convergence Rates, Acceleration, and Applicationscited by 2
  16. 2023Quasi-Newton Updating for Large-Scale Distributed Learningcited by 2
  17. 2021Distributed Sparse Regression via Penalizationcited by 1
  18. 2023Network-GIANT: Fully distributed Newton-type optimization via harmonic Hessian consensuscited by 1
  19. 2023Q-SHED: Distributed Optimization at the Edge via Hessian Eigenvectors Quantizationcited by 1
  20. 2023Optimal Compression of Unit Norm Vectors in the High Distortion Regimecited by 1
  21. 2024FAGH: Accelerating Federated Learning with Approximated Global Hessiancited by 1
  22. 2024Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasingcited by 1
  23. 2024GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learningcited by 1
  24. 2021Learning Linear Models Using Distributed Iterative Hessian Sketchingcited by 0
  25. 2024A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniquescited by 0
  26. 2024Efficient Federated Low Rank Matrix Completioncited by 0
  27. 2025FedOSAA: Improving Federated Learning with One-Step Anderson Accelerationcited by 0
  28. 2025Accelerated Training of Federated Learning via Second-Order Methodscited by 0
  29. 2025A Split-Client Approach to Second-Order Optimizationcited by 0
  30. 2026DP-FedSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrixcited by 0
  31. 2026SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneitycited by 0