# Communication-Efficient Federated Learning with Compensated Overlap-FedAvg

arXiv: https://arxiv.org/abs/2012.06706 · HTML: https://mlsanity.com/paper/2012.06706/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2020.

## Built on by (most cited first, 30 of 31)

- [Accelerated Federated Learning with Decoupled Adaptive Optimization](https://mlsanity.com/paper/2207.07223/) (2022)
- [FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clients](https://mlsanity.com/paper/2401.02433/) (2023)
- [Communication-Efficient Online Federated Learning Framework for Nonlinear Regression](https://mlsanity.com/paper/2110.06556/) (2021)
- [Federated Learning over Coupled Graphs](https://mlsanity.com/paper/2301.11099/) (2023)
- [From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization](https://mlsanity.com/paper/2112.09355/) (2021)
- [Optimising Communication Overhead in Federated Learning Using NSGA-II](https://mlsanity.com/paper/2204.02183/) (2022)
- [Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learning](https://mlsanity.com/paper/2401.08216/) (2024)
- [Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions](https://mlsanity.com/paper/2110.04160/) (2021)
- [Federated Learning for Cyber Physical Systems: A Comprehensive Survey](https://mlsanity.com/paper/2505.04873/) (2025)
- [Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence](https://mlsanity.com/paper/2302.13562/) (2023)
- [Matching DNN Compression and Cooperative Training with Resources and Data Availability](https://mlsanity.com/paper/2212.02304/) (2022)
- [Federated cINN Clustering for Accurate Clustered Federated Learning](https://mlsanity.com/paper/2309.01515/) (2023)
- [Ferret: An Efficient Online Continual Learning Framework under Varying Memory Constraints](https://mlsanity.com/paper/2503.12053/) (2025)
- [E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizing](https://mlsanity.com/paper/2502.03092/) (2025)
- [OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning](https://mlsanity.com/paper/2105.00602/) (2021)
- [PiPar: Pipeline Parallelism for Collaborative Machine Learning](https://mlsanity.com/paper/2302.12803/) (2022)
- [Federated Two Stage Decoupling With Adaptive Personalization Layers](https://mlsanity.com/paper/2308.15821/) (2023)
- [Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systems](https://mlsanity.com/paper/2404.13348/) (2024)
- [Federated Learning-Distillation Alternation for Resource-Constrained IoT](https://mlsanity.com/paper/2505.20456/) (2025)
- [Deploying Models to Non-participating Clients in Federated Learning without Fine-tuning: A Hypernetwork-based Approach](https://mlsanity.com/paper/2508.12673/) (2025)
- [Improving the Robustness of Federated Learning for Severely Imbalanced Datasets](https://mlsanity.com/paper/2204.13414/) (2022)
- [Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descent](https://mlsanity.com/paper/2209.11944/) (2022)
- [Personalized Federated Learning with Hidden Information on Personalized Prior](https://mlsanity.com/paper/2211.10684/) (2022)
- [Context-Aware Orchestration of Energy-Efficient Gossip Learning Schemes](https://mlsanity.com/paper/2404.12023/) (2024)
- [On-Demand Model and Client Deployment in Federated Learning with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.07175/) (2024)
- [Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization](https://mlsanity.com/paper/2405.17932/) (2024)
- [ForgeVLA: Federated Vision-Language-Action Learning without Language Annotations](https://mlsanity.com/paper/2605.07474/) (2026)
- [DeFTA: A Plug-and-Play Decentralized Replacement for FedAvg](https://mlsanity.com/paper/2204.02632/) (2022)
- [Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness](https://mlsanity.com/paper/2309.13536/) (2023)
- [Dependable Distributed Training of Compressed Machine Learning Models](https://mlsanity.com/paper/2402.14346/) (2024)

## Abstract (arXiv)

Petabytes of data are generated each day by emerging Internet of Things (IoT), but only few of them can be finally collected and used for Machine Learning (ML) purposes due to the apprehension of data & privacy leakage, which seriously retarding ML's growth. To alleviate this problem, Federated learning is proposed to perform model training by multiple clients' combined data without the dataset sharing within the cluster. Nevertheless, federated learning introduces massive communication overhead as the synchronized data in each epoch is of the same size as the model, and thereby leading to a low communication efficiency. Consequently, variant methods mainly focusing on the communication rounds reduction and data compression are proposed to reduce the communication overhead of federated learning. In this paper, we propose Overlap-FedAvg, a framework that parallels the model training phase with model uploading & downloading phase, so that the latter phase can be totally covered by the former phase. Compared to vanilla FedAvg, Overlap-FedAvg is further developed with a hierarchical computing strategy, a data compensation mechanism and a nesterov accelerated gradients~(NAG) algorithm. Besides, Overlap-FedAvg is orthogonal to many other compression methods so that they can be applied together to maximize the utilization of the cluster. Furthermore, the theoretical analysis is provided to prove the convergence of the proposed Overlap-FedAvg framework. Extensive experiments on both conventional and recurrent tasks with multiple models and datasets also demonstrate that the proposed Overlap-FedAvg framework substantially boosts the federated learning process.