mlsanityData to 14 Oct 2026

Communication-Efficient Federated Learning with Accelerated Client Gradient

arXiv 2201.03172 ↗First posted 10 Jan 2022cs.LG cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

Federated learning often suffers from slow and unstable convergence due to the heterogeneous characteristics of participating client datasets. Such a tendency is aggravated when the client participation ratio is low since the information collected from the clients has large variations. To address this challenge, we propose a simple but effective federated learning framework, which improves the consistency across clients and facilitates the convergence of the server model. This is achieved by making the server broadcast a global model with a lookahead gradient. This strategy enables the proposed approach to convey the projected global update information to participants effectively without additional client memory and extra communication costs. We also regularize local updates by aligning each client with the overshot global model to reduce bias and improve the stability of our algorithm. We provide the theoretical convergence rate of our algorithm and demonstrate remarkable performance gains in terms of accuracy and communication efficiency compared to the state-of-the-art methods, especially with low client participation rates. The source code is available at our project page.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 34 of its 52 references that are in the map. First posted on arXiv 10 Jan 2022.

Builds on

34 of 52 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2018Federated Learning with Non-IID Datacited by 1,108
  9. 2020Adaptive Federated Optimizationcited by 1,046
  10. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  11. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  12. 2018LEAF: A Benchmark for Federated Settingscited by 792
22 more
  1. 2021Model-Contrastive Federated Learningcited by 766
  2. 2018Local SGD Converges Fast and Communicates Littlecited by 605
  3. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  4. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  5. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  6. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  7. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 306
  8. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  9. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  10. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 181
  11. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
  12. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  13. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  14. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  15. 2019First Analysis of Local GD on Heterogeneous Datacited by 81
  16. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 81
  17. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  18. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  19. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  20. 2020Faster Non-Convex Federated Learning via Global and Local Momentumcited by 48
  21. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  22. 2020FedADC: Accelerated Federated Learning with Drift Controlcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 15; 2026: 3.

24 papers.

Sort:

By area:

  1. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  2. 2025FedSWA: Improving Generalization in Federated Learning with Highly Heterogeneous Data via Momentum-Based Stochastic Controlled Weight Averagingcited by 25
  3. 2023Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentumcited by 3
  4. 2024SCALA: Split Federated Learning with Concatenated Activations and Logit Adjustmentscited by 3
  5. 2025GC-Fed: Gradient Centralized Federated Learning with Partial Client Participationcited by 2
  6. 2023Adaptive Self-Distillation for Minimizing Client Drift in Heterogeneous Federated Learningcited by 1
  7. 2024Communication-Efficient Federated Learning under Dynamic Device Arrival and Departure: Convergence Analysis and Algorithm Designcited by 1
  8. 2025Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learningcited by 1
  9. 2025Federated Self-Supervised Learning for One-Shot Cross-Modal and Cross-Imaging Technique Segmentationcited by 1
  10. 2025Towards Unified Modeling in Federated Multi-Task Learning via Subspace Decouplingcited by 1
  11. 2025FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantizationcited by 1
  12. 2025FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fieldscited by 1
12 more
  1. 2025DictPFL: Efficient and Private Federated Learning on Encrypted Gradientscited by 1
  2. 2024Neighborhood and Global Perturbations Supported SAM in Federated Learning: From Local Tweaks To Global Awarenesscited by 0
  3. 2025FedPalm: A General Federated Learning Framework for Closed- and Open-Set Palmprint Verificationcited by 0
  4. 2025TACO: Tackling Over-correction in Federated Learning with Tailored Adaptive Correctioncited by 0
  5. 2025Hybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learningcited by 0
  6. 2025Decoupled Contrastive Learning for Federated Learningcited by 0
  7. 2025Generalizable Federated Learning using Client Adaptive Focal Modulationcited by 0
  8. 2025On the Limits of Momentum in Decentralized and Federated Optimizationcited by 0
  9. 2025Federated Learning with Feedback Alignmentcited by 0
  10. 2026Federated Learning of Binary Neural Networks: Enabling Low-Cost Inferencecited by 0
  11. 2026FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learningcited by 0
  12. 2026FedACT: Federated Adaptive Coordinate Trust Modulation for Robust Transformer Training under Data Heterogeneitycited by 0