mlsanityData to 7 Oct 2026

Communication-Efficient Adaptive Federated Learning

arXiv 2205.02719 ↗First posted 5 May 2022cs.LG cs.AI cs.DCcited by 55

Not on the map. Why?

Abstract (arXiv)

Federated learning is a machine learning training paradigm that enables clients to jointly train models without sharing their own localized data. However, the implementation of federated learning in practice still faces numerous challenges, such as the large communication overhead due to the repetitive server-client synchronization and the lack of adaptivity by SGD-based model updates. Despite that various methods have been proposed for reducing the communication cost by gradient compression or quantization, and the federated versions of adaptive optimizers such as FedAdam are proposed to add more adaptivity, the current federated learning framework still cannot solve the aforementioned challenges all at once. In this paper, we propose a novel communication-efficient adaptive federated learning method (FedCAMS) with theoretical convergence guarantees. We show that in the nonconvex stochastic optimization setting, our proposed FedCAMS achieves the same convergence rate of O(frac1√TKm) as its non-compressed counterparts. Extensive experiments on various benchmarks verify our theoretical analysis.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 5 May 2022.

Builds on

40 of 56 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  10. 2020Adaptive Federated Optimizationcited by 1,036
  11. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  12. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
28 more
  1. 2019On the Convergence of Adam and Beyondcited by 834
  2. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  3. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  4. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  5. 2019Fair Resource Allocation in Federated Learningcited by 389
  6. 2018Sparsified SGD with Memorycited by 373
  7. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  8. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  9. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  10. 2018The Convergence of Sparsified Gradient Methodscited by 228
  11. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  12. 2022Patches Are All You Need?cited by 201
  13. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  14. 2019Adaptive Gradient Methods with Dynamic Bound of Learning Ratecited by 181
  15. 2019Energy Efficient Federated Learning Over Wireless Communication Networkscited by 181
  16. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  17. 2018On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimizationcited by 173
  18. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  19. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  20. 2018On the Convergence of Adaptive Gradient Methods for Nonconvex Optimizationcited by 91
  21. 2018Closing the Generalization Gap of Adaptive Gradient Methods in Training Deep Neural Networkscited by 89
  22. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 88
  23. 20211-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speedcited by 74
  24. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  25. 2020Quantized Adam with Error Feedbackcited by 27
  26. 2021Toward Communication Efficient Adaptive Gradient Methodcited by 25
  27. 2020Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Datacited by 15
  28. 2021Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimizationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 12; 2024: 19; 2025: 15; 2026: 6.

55 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2022Faster Adaptive Federated Learningcited by 35
  3. 2024Relaxed Contrastive Learning for Federated Learningcited by 32
  4. 2023On the Vulnerability of Backdoor Defenses for Federated Learningcited by 27
  5. 2023GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgettingcited by 27
  6. 2023FS-Real: Towards Real-World Cross-Device Federated Learningcited by 22
  7. 2022Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learningcited by 14
  8. 2023FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systemscited by 14
  9. 2024Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimizationcited by 12
  10. 2024FADAS: Towards Federated Adaptive Asynchronous Optimizationcited by 10
  11. 2022Communication-Efficient Federated Learning With Data and Client Heterogeneitycited by 9
  12. 2024Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resiliencecited by 8
38 more
  1. 2024FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injectioncited by 8
  2. 2024Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuningcited by 8
  3. 2023Locally Adaptive Federated Learningcited by 7
  4. 2025Efficient Distributed Optimization under Heavy-Tailed Noisecited by 7
  5. 2025Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Accelerationcited by 7
  6. 2024FedLion: Faster Adaptive Federated Optimization with Fewer Communicationcited by 6
  7. 2024Blockchain-empowered Federated Learning: Benefits, Challenges, and Solutionscited by 5
  8. 2024FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learningcited by 5
  9. 2024CSI-GPT: Integrating Generative Pre-Trained Transformer with Federated-Tuning to Acquire Downlink Massive MIMO Channelscited by 5
  10. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  11. 2023Federated Learning: A Cutting-Edge Survey of the Latest Advancements and Applicationscited by 3
  12. 2024An Element-Wise Weights Aggregation Method for Federated Learningcited by 3
  13. 2024Federated PCA on Grassmann Manifold for IoT Anomaly Detectioncited by 3
  14. 2025Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimizationcited by 3
  15. 2023Secure and Fast Asynchronous Vertical Federated Learning via Cascaded Hybrid Optimizationcited by 2
  16. 2023FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Datacited by 2
  17. 2023Communication-Efficient Federated Optimization over Semi-Decentralized Networkscited by 2
  18. 2023Robust Collaborative Inference with Vertically Split Data Over Dynamic Device Environmentscited by 2
  19. 2024Efficient Adaptive Federated Optimizationcited by 2
  20. 2026Rethinking LoRA for Data Heterogeneous Federated Learning: Subspace and State Alignmentcited by 2
  21. 2023A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averagingcited by 1
  22. 2024Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimizationcited by 1
  23. 2024Peak-Controlled Logits Poisoning Attack in Federated Distillationcited by 1
  24. 2024Personalized Federated Learning for Generative AI-Assisted Semantic Communicationscited by 1
  25. 2024Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guaranteescited by 1
  26. 2025Gradient Correction in Federated Learning with Adaptive Optimizationcited by 1
  27. 2025FedABC: Attention-Based Client Selection for Federated Learning with Long-Term Viewcited by 1
  28. 2025Communication-Efficient Zero-Order and First-Order Federated Learning Methods over Wireless Networkscited by 1
  29. 2025Time-varying Mixing Matrix Design for Energy-efficient Decentralized Federated Learningcited by 1
  30. 2024Painless Federated Learning: An Interplay of Line-Search and Extrapolationcited by 0
  31. 2025Communication-Efficient Federated Learning Based on Explanation-Guided Pruning for Remote Sensing Image Classificationcited by 0
  32. 2025Federated Koopman-Reservoir Learning for Large-Scale Multivariate Time-Series Anomaly Detectioncited by 0
  33. 2025Analysis of Asynchronous Federated Learning: Unraveling the Interactions between Gradient Compression, Delay, and Data Heterogeneitycited by 0
  34. 2025Event-Driven Online Vertical Federated Learningcited by 0
  35. 2025FedReplay: A Feature Replay Assisted Federated Transfer Learning Framework for Efficient and Privacy-Preserving Smart Agriculturecited by 0
  36. 2025Why Federated Optimization Fails to Achieve Perfect Fitting? A Theoretical Perspective on Client-Side Optimacited by 0
  37. 2025ParaBlock: Communication-Computation Parallel Block Coordinate Federated Learning for Large Language Modelscited by 0
  38. 2025Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learningcited by 0

50 most cited of 55 All 55 (JSON)