# Fast-Convergent Federated Learning

arXiv: https://arxiv.org/abs/2007.13137 · HTML: https://mlsanity.com/paper/2007.13137/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2020.

## Built on by (most cited first, 30 of 61)

- [Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling](https://mlsanity.com/paper/2112.11256/) (2021)
- [Client Selection in Federated Learning: Principles, Challenges, and Opportunities](https://mlsanity.com/paper/2211.01549/) (2022)
- [DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning](https://mlsanity.com/paper/2111.00465/) (2021)
- [Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation](https://mlsanity.com/paper/2101.00787/) (2021)
- [Communication Efficiency in Federated Learning: Achievements and Challenges](https://mlsanity.com/paper/2107.10996/) (2021)
- [Node Selection Toward Faster Convergence for Federated Learning on Non-IID Data](https://mlsanity.com/paper/2105.07066/) (2021)
- [On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond](https://mlsanity.com/paper/2206.05187/) (2022)
- [Cost-Effective Federated Learning in Mobile Edge Networks](https://mlsanity.com/paper/2109.05411/) (2021)
- [A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency](https://mlsanity.com/paper/2307.10655/) (2023)
- [Context-Aware Online Client Selection for Hierarchical Federated Learning](https://mlsanity.com/paper/2112.00925/) (2021)
- [A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guarantee](https://mlsanity.com/paper/2108.00365/) (2021)
- [Communication-Efficient Online Federated Learning Framework for Nonlinear Regression](https://mlsanity.com/paper/2110.06556/) (2021)
- [Decentralized Unsupervised Learning of Visual Representations](https://mlsanity.com/paper/2111.10763/) (2021)
- [Resource Constrained Vehicular Edge Federated Learning with Highly Mobile Connected Vehicles](https://mlsanity.com/paper/2210.15496/) (2022)
- [Federated Learning with Regularized Client Participation](https://mlsanity.com/paper/2302.03662/) (2023)
- [Efficient Federated Meta-Learning over Multi-Access Wireless Networks](https://mlsanity.com/paper/2108.06453/) (2021)
- [Federated Learning and Meta Learning: Approaches, Applications, and Directions](https://mlsanity.com/paper/2210.13111/) (2022)
- [Green Federated Learning: A new era of Green Aware AI](https://mlsanity.com/paper/2409.12626/) (2024)
- [Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds](https://mlsanity.com/paper/2204.03726/) (2022)
- [Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networks](https://mlsanity.com/paper/2404.13804/) (2024)
- [To Store or Not? Online Data Selection for Federated Learning with Limited Storage](https://mlsanity.com/paper/2209.00195/) (2022)
- [Addressing Heterogeneity in Federated Learning via Distributional Transformation](https://mlsanity.com/paper/2210.15025/) (2022)
- [Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective](https://mlsanity.com/paper/2411.07523/) (2024)
- [Client Selection for Federated Bayesian Learning](https://mlsanity.com/paper/2212.05492/) (2022)
- [Asynchronous Wireless Federated Learning with Probabilistic Client Selection](https://mlsanity.com/paper/2311.16741/) (2023)
- [Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective](https://mlsanity.com/paper/2405.20431/) (2024)
- [Adaptive Parameterization of Deep Learning Models for Federated Learning](https://mlsanity.com/paper/2302.02949/) (2023)
- [Adaptive Federated Learning via New Entropy Approach](https://mlsanity.com/paper/2303.14966/) (2023)
- [Resource-Efficient Federated Hyperdimensional Computing](https://mlsanity.com/paper/2306.01339/) (2023)
- [FedCode: Communication-Efficient Federated Learning via Transferring Codebooks](https://mlsanity.com/paper/2311.09270/) (2023)

## Abstract (arXiv)

Federated learning has emerged recently as a promising solution for distributing machine learning tasks through modern networks of mobile devices. Recent studies have obtained lower bounds on the expected decrease in model loss that is achieved through each round of federated learning. However, convergence generally requires a large number of communication rounds, which induces delay in model training and is costly in terms of network resources. In this paper, we propose a fast-convergent federated learning algorithm, called FOLB, which performs intelligent sampling of devices in each round of model training to optimize the expected convergence speed. We first theoretically characterize a lower bound on improvement that can be obtained in each round if devices are selected according to the expected improvement their local models will provide to the current global model. Then, we show that FOLB obtains this bound through uniform sampling by weighting device updates according to their gradient information. FOLB is able to handle both communication and computation heterogeneity of devices by adapting the aggregations according to estimates of device's capabilities of contributing to the updates. We evaluate FOLB in comparison with existing federated learning algorithms and experimentally show its improvement in trained model accuracy, convergence speed, and/or model stability across various machine learning tasks and datasets.