mlsanityData to 14 Oct 2026

Communication-Efficient Federated Learning with Compensated Overlap-FedAvg

arXiv 2012.06706 ↗First posted 12 Dec 2020cs.LG cs.DCcited by 31

Not on the map. Why?

Abstract (arXiv)

Petabytes of data are generated each day by emerging Internet of Things (IoT), but only few of them can be finally collected and used for Machine Learning (ML) purposes due to the apprehension of data & privacy leakage, which seriously retarding ML's growth. To alleviate this problem, Federated learning is proposed to perform model training by multiple clients' combined data without the dataset sharing within the cluster. Nevertheless, federated learning introduces massive communication overhead as the synchronized data in each epoch is of the same size as the model, and thereby leading to a low communication efficiency. Consequently, variant methods mainly focusing on the communication rounds reduction and data compression are proposed to reduce the communication overhead of federated learning. In this paper, we propose Overlap-FedAvg, a framework that parallels the model training phase with model uploading & downloading phase, so that the latter phase can be totally covered by the former phase. Compared to vanilla FedAvg, Overlap-FedAvg is further developed with a hierarchical computing strategy, a data compensation mechanism and a nesterov accelerated gradients~(NAG) algorithm. Besides, Overlap-FedAvg is orthogonal to many other compression methods so that they can be applied together to maximize the utilization of the cluster. Furthermore, the theoretical analysis is provided to prove the convergence of the proposed Overlap-FedAvg framework. Extensive experiments on both conventional and recurrent tasks with multiple models and datasets also demonstrate that the proposed Overlap-FedAvg framework substantially boosts the federated learning process.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 6; 2024: 6; 2025: 5; 2026: 2.

31 papers.

Sort:

By area:

  1. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  2. 2023FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clientscited by 23
  3. 2021Communication-Efficient Online Federated Learning Framework for Nonlinear Regressioncited by 16
  4. 2023Federated Learning over Coupled Graphscited by 12
  5. 2021From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimizationcited by 8
  6. 2022Optimising Communication Overhead in Federated Learning Using NSGA-IIcited by 7
  7. 2024Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learningcited by 7
  8. 2021Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directionscited by 6
  9. 2025Federated Learning for Cyber Physical Systems: A Comprehensive Surveycited by 6
  10. 2023Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergencecited by 5
  11. 2022Matching DNN Compression and Cooperative Training with Resources and Data Availabilitycited by 4
  12. 2023Federated cINN Clustering for Accurate Clustered Federated Learningcited by 4
19 more
  1. 2025Ferret: An Efficient Online Continual Learning Framework under Varying Memory Constraintscited by 4
  2. 2025E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizingcited by 3
  3. 2021OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learningcited by 2
  4. 2022PiPar: Pipeline Parallelism for Collaborative Machine Learningcited by 2
  5. 2023Federated Two Stage Decoupling With Adaptive Personalization Layerscited by 2
  6. 2024Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systemscited by 2
  7. 2025Federated Learning-Distillation Alternation for Resource-Constrained IoTcited by 2
  8. 2025Deploying Models to Non-participating Clients in Federated Learning without Fine-tuning: A Hypernetwork-based Approachcited by 2
  9. 2022Improving the Robustness of Federated Learning for Severely Imbalanced Datasetscited by 1
  10. 2022Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descentcited by 1
  11. 2022Personalized Federated Learning with Hidden Information on Personalized Priorcited by 1
  12. 2024Context-Aware Orchestration of Energy-Efficient Gossip Learning Schemescited by 1
  13. 2024On-Demand Model and Client Deployment in Federated Learning with Deep Reinforcement Learningcited by 1
  14. 2024Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimizationcited by 1
  15. 2026ForgeVLA: Federated Vision-Language-Action Learning without Language Annotationscited by 1
  16. 2022DeFTA: A Plug-and-Play Decentralized Replacement for FedAvgcited by 0
  17. 2023Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Stalenesscited by 0
  18. 2024Dependable Distributed Training of Compressed Machine Learning Modelscited by 0
  19. 2026Accelerating Decentralized Optimization via Overlapping Local Stepscited by 0