mlsanityData to 7 Oct 2026

FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learning

arXiv 2206.08829 ↗First posted 17 Jun 2022cs.LG cs.CR cs.DCcited by 23

Not on the map. Why?

Abstract (arXiv)

Newton-type methods are popular in federated learning due to their fast convergence. Still, they suffer from two main issues, namely: low communication efficiency and low privacy due to the requirement of sending Hessian information from clients to parameter server (PS). In this work, we introduced a novel framework called FedNew in which there is no need to transmit Hessian information from clients to PS, hence resolving the bottleneck to improve communication efficiency. In addition, FedNew hides the gradient information and results in a privacy-preserving approach compared to the existing state-of-the-art. The core novel idea in FedNew is to introduce a two level framework, and alternate between updating the inverse Hessian-gradient product using only one alternating direction method of multipliers (ADMM) step and then performing the global model update using Newton's method. Though only one ADMM pass is used to approximate the inverse Hessian-gradient product at each iteration, we develop a novel theoretical approach to show the converging behavior of FedNew for convex problems. Additionally, a significant reduction in communication overhead is achieved by utilizing stochastic quantization. Numerical results using real datasets show the superiority of FedNew compared to existing methods in terms of communication costs.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 17 Jun 2022.

Builds on

21 of 33 references.

References · 21

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  3. 2020Adaptive Federated Optimizationcited by 1,036
  4. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  5. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  6. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  7. 2019SGD: General Analysis and Improved Ratescited by 206
  8. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 95
  9. 2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
  10. 2019Adaptive Gradient Descent without Descentcited by 79
  11. 2018Distributed learning with compressed gradientscited by 53
  12. 2021FedNL: Making Newton-Type Methods Applicable to Federated Learningcited by 50
9 more
  1. 2018An Exact Quantized Decentralized Gradient Descent Algorithmcited by 34
  2. 2021Proximal and Federated Random Reshufflingcited by 33
  3. 2019Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradientscited by 32
  4. 2019Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Ratescited by 26
  5. 2018Global linear convergence of Newton's method without strong-convexity or Lipschitz gradientscited by 24
  6. 2021Regularized Newton Method with Global O(1/k2) Convergencecited by 22
  7. 2019Globally Convergent Newton Methods for Ill-conditioned Generalized Self-concordant Lossescited by 17
  8. 2019Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learningcited by 17
  9. 2020Convergence and Complexity Analysis of a Levenberg-Marquardt Algorithm for Inverse Problemscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 11; 2025: 7; 2026: 1.

23 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2023Bayesian Federated Learning: A Surveycited by 22
  3. 2023Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updatescited by 20
  4. 2024Federated Combinatorial Multi-Agent Multi-Armed Banditscited by 11
  5. 2025Tackling Feature and Sample Heterogeneity in Decentralized Multi-Task Learning: A Sheaf-Theoretic Approachcited by 10
  6. 2025FedMuon: Federated Learning with Bias-corrected LMO-based Optimizationcited by 7
  7. 2024Lightweight Unsupervised Federated Learning with Pretrained Vision Language Modelcited by 5
  8. 2024Fed-Sophia: A Communication-Efficient Second-Order Federated Learning Algorithmcited by 5
  9. 2024Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Methodcited by 4
  10. 2025Distributed Quasi-Newton Method for Fair and Fast Federated Learningcited by 4
  11. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  12. 2025Federated Adapter on Foundation Models: An Out-Of-Distribution Approachcited by 2
11 more
  1. 2022Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topologycited by 1
  2. 2024Communication and Energy Efficient Federated Learning using Zero-Order Optimization Techniquecited by 1
  3. 2024A Novel Framework of Horizontal-Vertical Hybrid Federated Learning for EdgeIoTcited by 1
  4. 2024Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregationcited by 1
  5. 2025Communication-Efficient Zero-Order and First-Order Federated Learning Methods over Wireless Networkscited by 1
  6. 2025Energy and Memory-Efficient Federated Learning With Ordered Layer Freezingcited by 1
  7. 2024Painless Federated Learning: An Interplay of Line-Search and Extrapolationcited by 0
  8. 2024FLeNS: Federated Learning with Enhanced Nesterov-Newton Sketchcited by 0
  9. 2024Privacy-Preserving Federated Unsupervised Domain Adaptation for Regression on Small-Scale and High-Dimensional Biological Datacited by 0
  10. 2025Accelerated Training of Federated Learning via Second-Order Methodscited by 0
  11. 2026Multi-Agent Combinatorial-Multi-Armed-Bandit framework for the Submodular Welfare Problem under Bandit Feedbackcited by 0