mlsanityData to 7 Oct 2026

FedSSO: A Federated Server-Side Second-Order Optimization Algorithm

arXiv 2206.09576 ↗First posted 20 Jun 2022cs.LG cs.AI math.OCcited by 3

Not on the map. Why?

Abstract (arXiv)

In this work, we propose FedSSO, a server-side second-order optimization method for federated learning (FL). In contrast to previous works in this direction, we employ a server-side approximation for the Quasi-Newton method without requiring any training data from the clients. In this way, we not only shift the computation burden from clients to server, but also eliminate the additional communication for second-order updates between clients and server entirely. We provide theoretical guarantee for convergence of our novel method, and empirically demonstrate our fast convergence and communication savings in both convex and non-convex settings.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 71 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

43 of 71 references.

References · 43

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2020Adaptive Federated Optimizationcited by 1,036
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  11. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  12. 2019Federated Learning with Personalization Layerscited by 533
31 more
  1. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  2. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  3. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  4. 2019SecureBoost: A Lossless Federated Learning Frameworkcited by 174
  5. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  6. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  7. 2013Communication Efficient Distributed Optimization using an Approximate Newton-type Methodcited by 131
  8. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  9. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  10. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  11. 2014A Stochastic Quasi-Newton Method for Large-Scale Optimizationcited by 68
  12. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 62
  13. 2015Communication Complexity of Distributed Convex Learning and Optimizationcited by 59
  14. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  15. 2017GIANT: Globally Improved Approximate Newton Method for Distributed Optimizationcited by 43
  16. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  17. 2016AIDE: Fast and Communication Efficient Distributed Optimizationcited by 42
  18. 2015A Linearly-Convergent Stochastic L-BFGS Algorithmcited by 39
  19. 2016Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimizationcited by 37
  20. 2021Distributed Second Order Methods with Fast Rates and Compressed Communicationcited by 25
  21. 2019Communication-Efficient Local Decentralized SGD Methodscited by 21
  22. 2019Distributed estimation of the inverse Hessian by determinantal averagingcited by 16
  23. 2014RES: Regularized Stochastic BFGS Algorithmcited by 15
  24. 2021LocalNewton: Reducing Communication Bottleneck for Distributed Learningcited by 15
  25. 2021Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learningcited by 14
  26. 2018Fast determinantal point processes via distortion-free intermediate samplingcited by 13
  27. 2013Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methodscited by 9
  28. 2021On Second-order Optimization Methods for Federated Learningcited by 9
  29. 2020Second-Order Guarantees in Federated Learningcited by 5
  30. 2018Correcting the bias in least squares regression with volume-rescaled samplingcited by 3
  31. 2018Implementation of Stochastic Quasi-Newton's Method in PyTorchcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
  2. 2024FAGH: Accelerating Federated Learning with Approximated Global Hessiancited by 1
  3. 2025Accelerated Training of Federated Learning via Second-Order Methodscited by 0