FedFetch: Faster Federated Learning with Adaptive Downstream Prefetching
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) is a machine learning paradigm that facilitates massively distributed model training with end-user data on edge devices directed by a central server. However, the large number of heterogeneous clients in FL deployments leads to a communication bottleneck between the server and the clients. This bottleneck is made worse by straggling clients, any one of which will further slow down training. To tackle these challenges, researchers have proposed techniques like client sampling and update compression. These techniques work well in isolation but combine poorly in the downstream, server-to-client direction. This is because unselected clients have outdated local model states and need to synchronize these states with the server first. We introduce FedFetch, a strategy to mitigate the download time overhead caused by combining client sampling and compression techniques. FedFetch achieves this with an efficient prefetch schedule for clients to prefetch model states multiple rounds before a stated training round. We empirically show that adding FedFetch to communication efficient FL techniques reduces end-to-end training time by 1.26× and download time by 4.49× across compression techniques with heterogeneous client settings. Our implementation is available at https://github.com/DistributedML/FedFetch
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 21 Apr 2025.
Builds on
25 of 40 references.
References · 25
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
- 2019Towards Federated Learning at Scale: System Designcited by 826
- 2018LEAF: A Benchmark for Federated Settingscited by 780
- 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
- 2018Sparsified SGD with Memorycited by 373
- 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
- 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
13 more
- 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
- 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
- 2020Optimal Client Sampling for Federated Learningcited by 123
- 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
- 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
- 2020Federated Learning With Quantized Global Model Updatescited by 54
- 2021EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learningcited by 53
- 2020Design and Analysis of Uplink and Downlink Communications for Federated Learningcited by 48
- 2023DoCoFL: Downlink Compression for Cross-Device Federated Learningcited by 27
- 2024FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extractioncited by 21
- 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
- 2022GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learningcited by 7
- 2022Sign Bit is Enough: A Learning Synchronization Framework for Multi-hop All-reduce with Ultimate Compressioncited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.