mlsanityData to 7 Oct 2026

Communication Efficient Distributed Optimization using an Approximate Newton-type Method

arXiv 1312.7853 ↗First posted 30 Dec 2013cs.LG math.OC stat.MLcited by 131

Not on the map. Why?

Abstract (arXiv)

We present a novel Newton-type method for distributed optimization, which is particularly well suited for stochastic optimization and learning problems. For quadratic objectives, the method enjoys a linear rate of convergence which provably improves with the data size, requiring an essentially constant number of iterations under reasonable assumptions. We provide theoretical and empirical evidence of the advantages of our method compared to other approaches, such as one-shot parameter averaging and ADMM.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Dec 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 40; 2022: 30; 2023: 23; 2024: 21; 2025: 13; 2026: 4.

131 papers.

Sort:

  1. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  2. 2021A Field Guide to Federated Optimizationcited by 312
  3. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  4. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  5. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  6. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
  7. 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47
  8. 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
  9. 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
  10. 2021Node Selection Toward Faster Convergence for Federated Learning on Non-IID Datacited by 39
  11. 2022On-Demand Sampling: Learning Optimally from Multiple Distributionscited by 39
  12. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
38 more
  1. 2021The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communicationcited by 30
  2. 2022Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraintscited by 30
  3. 2021FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimizationcited by 29
  4. 2023Asynchronous Federated Continual Learningcited by 29
  5. 2022FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulationscited by 27
  6. 2021Distributed Second Order Methods with Fast Rates and Compressed Communicationcited by 25
  7. 2021Decentralized Local Stochastic Extra-Gradient for Variational Inequalitiescited by 25
  8. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  9. 2023Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reductioncited by 23
  10. 2021Prototype Guided Federated Learning of Visual Feature Representationscited by 22
  11. 2022AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimationcited by 21
  12. 2023Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updatescited by 20
  13. 2022SphereFed: Hyperspherical Federated Learningcited by 19
  14. 2022Faster federated optimization under second-order similaritycited by 18
  15. 2021Acceleration in Distributed Optimization under Similaritycited by 17
  16. 2021FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systemscited by 16
  17. 2023Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oraclescited by 16
  18. 2021Differential Privacy Meets Federated Learning under Communication Constraintscited by 15
  19. 2021LocalNewton: Reducing Communication Bottleneck for Distributed Learningcited by 15
  20. 2021Adaptive Newton Sketch: Linear-time Optimization with Quadratic Convergence and Effective Hessian Dimensionalitycited by 14
  21. 2021Asynchronous Federated Learning for Sensor Data with Concept Driftcited by 14
  22. 2021Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learningcited by 14
  23. 2022Efficient Convex Optimization Requires Superlinear Memorycited by 14
  24. 2021Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradientscited by 13
  25. 2021Joint Optimization of Communications and Federated Learning Over the Aircited by 13
  26. 2022Distributed Learning of Generalized Linear Causal Networkscited by 13
  27. 2022Optimization with Access to Auxiliary Informationcited by 12
  28. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  29. 2021Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inferencecited by 11
  30. 2022SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharingcited by 11
  31. 2022Over-the-Air Federated Learning via Second-Order Optimizationcited by 11
  32. 2022Federated Random Reshuffling with Compression and Variance Reductioncited by 11
  33. 2022Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregationcited by 11
  34. 2023Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysiscited by 11
  35. 2021CSAFL: A Clustered Semi-Asynchronous Federated Learning Frameworkcited by 10
  36. 2021FedLGA: Towards System-Heterogeneity of Federated Learning via Local Gradient Approximationcited by 10
  37. 2024Federated Optimization with Doubly Regularized Drift Correctioncited by 10
  38. 2021On Second-order Optimization Methods for Federated Learningcited by 9

50 most cited of 131 All 131 (JSON)