mlsanityData to 14 Oct 2026

TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Training

arXiv 2304.06947 ↗First posted 14 Apr 2023cs.LG cs.DCcited by 27

Not on the map. Why?

Abstract (arXiv)

In cross-device Federated Learning (FL) environments, scaling synchronous FL methods is challenging as stragglers hinder the training process. Moreover, the availability of each client to join the training is highly variable over time due to system heterogeneities and intermittent connectivity. Recent asynchronous FL methods (e.g., FedBuff) have been proposed to overcome these issues by allowing slower users to continue their work on local training based on stale models and to contribute to aggregation when ready. However, we show empirically that this method can lead to a substantial drop in training accuracy as well as a slower convergence rate. The primary reason is that fast-speed devices contribute to many more rounds of aggregation while others join more intermittently or not at all, and with stale model updates. To overcome this barrier, we propose TimelyFL, a heterogeneity-aware asynchronous FL framework with adaptive partial training. During the training, TimelyFL adjusts the local training workload based on the real-time resource capabilities of each client, aiming to allow more available clients to join in the global update without staleness. We demonstrate the performance benefits of TimelyFL by conducting extensive experiments on various datasets (e.g., CIFAR-10, Google Speech, and Reddit) and models (e.g., ResNet20, VGG11, and ALBERT). In comparison with the state-of-the-art (i.e., FedBuff), our evaluations reveal that TimelyFL improves participation rate by 21.13%, harvests 1.28x - 2.89x more efficiency on convergence rate, and provides a 6.25% increment on test accuracy.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 14 Apr 2023.

Builds on

28 of 40 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  6. 2020Adaptive Federated Optimizationcited by 1,046
  7. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  8. 2021A Field Guide to Federated Optimizationcited by 315
  9. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
  10. 2019Asynchronous Federated Optimizationcited by 267
  11. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 190
  12. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
16 more
  1. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 90
  2. 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
  3. 2019AI Benchmark: All About Deep Learning on Smartphones in 2019cited by 77
  4. 2020MNN: A Universal and Efficient Inference Enginecited by 65
  5. 2021Federated Learning for Internet of Things: Applications, Challenges, and Opportunitiescited by 50
  6. 2021Layer-wise Adaptive Model Aggregation for Scalable Federated Learningcited by 36
  7. 2018Toward Understanding the Impact of Staleness in Distributed Machine Learningcited by 29
  8. 2022Scaling Language Model Size in Cross-Device Federated Learningcited by 25
  9. 2021Federated Learning for Internet of Things: A Federated Learning Framework for On-device Anomaly Data Detectioncited by 20
  10. 2019At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks?cited by 16
  11. 2022FedAudio: A Federated Learning Benchmark for Audio Taskscited by 16
  12. 2021Partial Variable Training for Efficient On-Device Federated Learningcited by 14
  13. 2021Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradientscited by 13
  14. 2021Efficient and Private Federated Learning with Partially Trainable Networkscited by 13
  15. 2021FedPrune: Towards Inclusive Federated Learningcited by 12
  16. 2022When Do Curricula Work in Federated Learning?cited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 13; 2026: 2.

27 papers.

Sort:

By area:

  1. 2024Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 49
  2. 2025HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Modelscited by 31
  3. 2024Asynchronous Local-SGD Training for Language Modelingcited by 26
  4. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  5. 2023FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategycited by 10
  6. 2025S2FGL: Spatial Spectral Federated Graph Learningcited by 7
  7. 2024Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integrationcited by 5
  8. 2024Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updatescited by 4
  9. 2024Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clientscited by 3
  10. 2025FedEL: Federated Elastic Learning for Heterogeneous Devicescited by 3
  11. 2025Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practicecited by 3
  12. 2023Federated Learning in IoT: a Survey from a Resource-Constrained Perspectivecited by 2
15 more
  1. 2025Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environmentscited by 2
  2. 2024CurvFed: Curvature-Aligned Federated Learning for Fairness without Demographicscited by 1
  3. 2025Enabling Weak Client Participation via On-device Knowledge Distillation in Heterogeneous Federated Learningcited by 1
  4. 2025Adaptive Deadline and Batch Layered Synchronized Federated Learningcited by 1
  5. 2023Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learningcited by 0
  6. 2024FedTSA: A Cluster-based Two-Stage Aggregation Method for Model-heterogeneous Federated Learningcited by 0
  7. 2024EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Modelscited by 0
  8. 2025SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Trainingcited by 0
  9. 2025Guided Model Merging for Hybrid Data Learning: Leveraging Centralized Data to Refine Decentralized Modelscited by 0
  10. 2025Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Trainingcited by 0
  11. 2025FedSA-GCL: A Semi-Asynchronous Federated Graph Learning Framework with Personalized Aggregation and Cluster-Aware Broadcastingcited by 0
  12. 2025PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecturecited by 0
  13. 2025Dynamic Participation in Federated Learning: Benchmarks and a Knowledge Pool Plugincited by 0
  14. 2026FedPSA: Modeling Behavioral Staleness in Asynchronous Federated Learningcited by 0
  15. 2026Robust Federated Learning Under Real-World Client Churncited by 0