mlsanityData to 7 Oct 2026

FedAgg: Adaptive Federated Learning with Aggregated Gradients

arXiv 2303.15799 ↗First posted 28 Mar 2023cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has emerged as a crucial distributed training paradigm, enabling discrete devices to collaboratively train a shared model under the coordination of a central server, while leveraging their locally stored private data. Nonetheless, the non-independent-and-identically-distributed (Non-IID) data generated on heterogeneous clients and the incessant information exchange among participants may significantly impede training efficacy, retard the model convergence rate and increase the risk of privacy leakage. To alleviate the divergence between the local and average model parameters and obtain a fast model convergence rate, we propose an adaptive FEDerated learning algorithm called FedAgg by refining the conventional stochastic gradient descent (SGD) methodology with an AGgregated Gradient term at each local training epoch and adaptively adjusting the learning rate based on a penalty term that quantifies the local model deviation. To tackle the challenge of information exchange among clients during local training and design a decentralized adaptive learning rate for each client, we introduce two mean-field terms to approximate the average local parameters and gradients over time. Through rigorous theoretical analysis, we demonstrate the existence and convergence of the mean-field terms and provide a robust upper bound on the convergence of our proposed algorithm. The extensive experimental results on real-world datasets substantiate the superiority of our framework in comparison with existing state-of-the-art FL strategies for enhancing model performance and accelerating convergence rate under IID and Non-IID datasets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 66 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

39 of 66 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2014Going Deeper with Convolutionscited by 4,231
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  9. 2020Adaptive Federated Optimizationcited by 1,036
  10. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  11. 2019Towards Federated Learning at Scale: System Designcited by 826
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
27 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  2. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  3. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  4. 2019Adaptive Gradient Methods with Dynamic Bound of Learning Ratecited by 181
  5. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  6. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  7. 2020Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspectivecited by 102
  8. 2020TiFL: A Tier-based Federated Learning Systemcited by 95
  9. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  10. 2020VAFL: a Method of Vertical Asynchronous Federated Learningcited by 87
  11. 2020Fast-Convergent Federated Learning with Adaptive Weightingcited by 67
  12. 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
  13. 2021HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noisecited by 61
  14. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  15. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  16. 2020Federated Learning With Quantized Global Model Updatescited by 54
  17. 2020Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activationscited by 47
  18. 2019Toward an Automated Auction Framework for Wireless Federated Learning Services Marketcited by 38
  19. 2020Optimal Gradient Compression for Distributed and Federated Learningcited by 37
  20. 2020Adaptive Federated Learning and Digital Twin for Industrial Internet of Thingscited by 27
  21. 2018AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methodscited by 24
  22. 2022Federated Learning with Position-Aware Neuronscited by 24
  23. 2022Scheduling and Aggregation Design for Asynchronous Federated Learning over Wireless Networkscited by 20
  24. 2022AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregationcited by 19
  25. 2023HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Associationcited by 17
  26. 2021FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systemscited by 16
  27. 2022Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropoutcited by 12

Built on by

Not cited yet.