mlsanityData to 7 Oct 2026

Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability

arXiv 2409.17446 ↗First posted 26 Sep 2024cs.DC cs.LG math.OCcited by 9

Not on the map. Why?

Abstract (arXiv)

Addressing intermittent client availability is critical for the real-world deployment of federated learning algorithms. Most prior work either overlooks the potential non-stationarity in the dynamics of client unavailability or requires substantial memory/computation overhead. We study federated learning in the presence of heterogeneous and non-stationary client availability, which may occur when the deployment environments are uncertain, or the clients are mobile. The impacts of heterogeneity and non-stationarity on client unavailability can be significant, as we illustrate using FedAvg, the most widely adopted federated learning algorithm. We propose FedAPM, which includes novel algorithmic structures that (i) compensate for missed computations due to unavailability with only O(1) additional memory and computation with respect to standard FedAvg, and (ii) evenly diffuse local updates within the federated learning system through implicit gossiping, despite being agnostic to non-stationary dynamics. We show that FedAPM converges to a stationary point of even non-convex objectives while achieving the desired linear speedup property. We corroborate our analysis with numerical experiments over diversified client unavailability dynamics on real-world data sets.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 46 of its 73 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

46 of 73 references.

References · 46

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  8. 2019Towards Federated Learning at Scale: System Designcited by 826
  9. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  10. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  11. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  12. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
34 more
  1. 2019Asynchronous Federated Optimizationcited by 264
  2. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  3. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  4. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  5. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  6. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  7. 2016Achieving Geometric Convergence for Distributed Optimization over Time-Varying Graphscited by 149
  8. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 125
  9. 2020Optimal Client Sampling for Federated Learningcited by 123
  10. 2017Network Topology and Communication-Computation Tradeoffs in Decentralized Optimizationcited by 119
  11. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  12. 2018D"IoT: A Federated Self-learning Anomaly Detection System for IoTcited by 89
  13. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  14. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 76
  15. 2021Fast Federated Learning in the Presence of Arbitrary Device Unavailabilitycited by 72
  16. 2020Towards Flexible Device Participation in Federated Learningcited by 64
  17. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 64
  18. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  19. 2022Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delayscited by 61
  20. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  21. 2021Anarchic Federated Learningcited by 42
  22. 2022On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyondcited by 37
  23. 2022Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compressioncited by 33
  24. 2023On the Convergence of Federated Averaging with Cyclic Client Participationcited by 31
  25. 2022Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraintscited by 30
  26. 2022Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimizationcited by 28
  27. 2021Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehiclescited by 26
  28. 2023How to Scale Your EMAcited by 22
  29. 2020Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availabilitycited by 20
  30. 2018Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functionscited by 19
  31. 2022Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregationcited by 11
  32. 2022Communication-Efficient Federated Learning With Data and Client Heterogeneitycited by 9
  33. 2023Towards Bias Correction of FedAvg over Nonuniform and Time-Varying Communicationscited by 6
  34. 2023Privacy-preserving and Uncertainty-aware Federated Trajectory Prediction for Connected Autonomous Vehiclescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 4.

9 papers.

  1. 2025Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainablecited by 6
  2. 2025FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Datacited by 4
  3. 2024Debiasing Federated Learning with Correlated Client Participationcited by 1
  4. 2025Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learningcited by 1
  5. 2025Unifying Physics- and Data-Driven Modeling via Novel Causal Spatiotemporal Graph Neural Network for Interpretable Epidemic Forecastingcited by 0
  6. 2026Federated Concept-Based Models: Interpretable models with distributed supervisioncited by 0
  7. 2026Agentic Federated Learning: The Future of Distributed Training Orchestrationcited by 0
  8. 2026Accelerating Optimization and Machine Learning through Decentralizationcited by 0
  9. 2026Federated Client Selection under Partial Visibility: A POMDP Approach with Spatio-Temporal Attentioncited by 0