mlsanityData to 14 Oct 2026

Queuing dynamics of asynchronous Federated Learning

arXiv 2405.00017 ↗First posted 12 Feb 2024cs.DC cs.LG stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

We study asynchronous federated learning mechanisms with nodes having potentially different computational speeds. In such an environment, each node is allowed to work on models with potential delays and contribute to updates to the central server at its own pace. Existing analyses of such algorithms typically depend on intractable quantities such as the maximum node delay and do not consider the underlying queuing dynamics of the system. In this paper, we propose a non-uniform sampling scheme for the central server that allows for lower delays with better complexity, taking into account the closed Jackson network structure of the associated computational graph. Our experiments clearly show a significant improvement of our method over current state-of-the-art asynchronous algorithms on an image classification problem.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 18 of its 52 references that are in the map. First posted on arXiv 12 Feb 2024.

Builds on

18 of 52 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  4. 2019Asynchronous Federated Optimizationcited by 267
  5. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 195
  6. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 190
  7. 2018Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptroncited by 143
  8. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  9. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 79
  10. 2022Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delayscited by 63
  11. 2023Optimal Time Complexities of Parallel Stochastic Optimization Methods Under a Fixed Computation Modelcited by 24
  12. 2021FedDQ: Communication-Efficient Federated Learning with Descending Quantizationcited by 20
6 more
  1. 2022AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregationcited by 19
  2. 2022A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updatescited by 17
  3. 2022DASHA: Distributed Nonconvex Optimization with Communication Compression, Optimal Oracle Complexity, and No Client Synchronizationcited by 16
  4. 2022Adaptive Compression for Communication-Efficient Distributed Trainingcited by 16
  5. 2023AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithmscited by 16
  6. 2023First Order Methods with Markovian Noise: from Acceleration to Variational Inequalitiescited by 14

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 2.

6 papers.

  1. 2024Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methodscited by 4
  2. 2024GAS: Generative Activation-Aided Asynchronous Split Federated Learningcited by 3
  3. 2024Asynchronous Federated Learning: A Scalable Approach for Decentralized Machine Learningcited by 3
  4. 2025Optimizing Asynchronous Federated Learning: A Delicate Trade-Off Between Model-Parameter Staleness and Update Frequencycited by 2
  5. 2024Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Datacited by 1
  6. 2025Guided Model Merging for Hybrid Data Learning: Leveraging Centralized Data to Refine Decentralized Modelscited by 0