mlsanityData to 14 Oct 2026

FedCAda: Adaptive Client-Side Optimization for Accelerated and Stable Federated Learning

arXiv 2405.11811 ↗First posted 20 May 2024cs.LG cs.DCcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) has emerged as a prominent approach for collaborative training of machine learning models across distributed clients while preserving data privacy. However, the quest to balance acceleration and stability becomes a significant challenge in FL, especially on the client-side. In this paper, we introduce FedCAda, an innovative federated client adaptive algorithm designed to tackle this challenge. FedCAda leverages the Adam algorithm to adjust the correction process of the first moment estimate m and the second moment estimate v on the client-side and aggregate adaptive algorithm parameters on the server-side, aiming to accelerate convergence speed and communication efficiency while ensuring stability and performance. Additionally, we investigate several algorithms incorporating different adjustment functions. This comparative analysis revealed that due to the limited information contained within client models from other clients during the initial stages of federated learning, more substantial constraints need to be imposed on the parameters of the adaptive algorithm. As federated learning progresses and clients gather more global information, FedCAda gradually diminishes the impact on adaptive parameters. These findings provide insights for enhancing the robustness and efficiency of algorithmic improvements. Through extensive experiments on computer vision (CV) and natural language processing (NLP) datasets, we demonstrate that FedCAda outperforms the state-of-the-art methods in terms of adaptability, convergence, stability, and overall performance. This work contributes to adaptive algorithms for federated learning, encouraging further exploration.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

25 of 36 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  8. 2019Advances and Open Problems in Federated Learningcited by 3,046
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  10. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  11. 2018Federated Learning with Non-IID Datacited by 1,108
  12. 2020Adaptive Federated Optimizationcited by 1,046
13 more
  1. 2019On the Convergence of Adam and Beyondcited by 841
  2. 2018LEAF: A Benchmark for Federated Settingscited by 792
  3. 2021Model-Contrastive Federated Learningcited by 766
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  5. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  6. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  7. 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
  8. 2020Federated Accelerated Stochastic Gradient Descentcited by 85
  9. 2023Revisiting Weighted Aggregation in Federated Learning with Neural Networkscited by 58
  10. 2022Communication-Efficient Adaptive Federated Learningcited by 55
  11. 2022Faster Adaptive Federated Learningcited by 35
  12. 2021On the Convergence of Decentralized Adaptive Gradient Methodscited by 17
  13. 2023Cuing Without Sharing: A Federated Cued Speech Recognition Framework via Mutual Knowledge Distillationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025MuLoCo: Muon is a practical inner optimizer for DiLoCocited by 18
  2. 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Reviewcited by 0
  3. 2025Adaptive Federated Learning via Dynamical System Modelcited by 0
  4. 2026ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edgecited by 0
  5. 2026FedZMG: Efficient Client-Side Optimization in Federated Learningcited by 0