mlsanityData to 14 Oct 2026

Quantized and Asynchronous Federated Learning

arXiv 2410.00242 ↗First posted 30 Sep 2024cs.LG eess.SP math.OCcited by 3

Not on the map. Why?

Abstract (arXiv)

Recent advances in federated learning have shown that asynchronous variants can be faster and more scalable than their synchronous counterparts. However, their design does not include quantization, which is necessary in practice to deal with the communication bottleneck. To bridge this gap, we develop a novel algorithm, Quantized Asynchronous Federated Learning (QAFeL), which introduces a hidden-state quantization scheme to avoid the error propagation caused by direct quantization. QAFeL also includes a buffer to aggregate client updates, ensuring scalability and compatibility with techniques such as secure aggregation. Furthermore, we prove that QAFeL achieves an O(1/√T) ergodic convergence rate for stochastic gradient descent on non-convex objectives, which is the optimal order of complexity, without requiring bounded gradients or uniform client arrivals. We also prove that the cross-term error between staleness and quantization only affects the higher-order error terms. We validate our theoretical findings on standard benchmarks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 34 of its 57 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

34 of 57 references.

References · 34

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2018Group Normalizationcited by 1,524
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  9. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  10. 2018LEAF: A Benchmark for Federated Settingscited by 792
  11. 2018Sparsified SGD with Memorycited by 376
  12. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
22 more
  1. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  2. 2014The Loss Surfaces of Multilayer Networkscited by 280
  3. 2019Asynchronous Federated Optimizationcited by 267
  4. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  5. 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 263
  6. 2018The Convergence of Sparsified Gradient Methodscited by 229
  7. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 221
  8. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 190
  9. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 133
  10. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
  11. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  12. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 90
  13. 2020A Better Alternative to Error Feedback for Communication-Efficient Distributed Learningcited by 43
  14. 2020Optimal Gradient Compression for Distributed and Federated Learningcited by 39
  15. 2022Machine Learning Model Sizes and the Parameter Gapcited by 39
  16. 2019The Complexity of Finding Stationary Points with Stochastic Gradient Descentcited by 35
  17. 2022Joint Privacy Enhancement and Quantization in Federated Learningcited by 35
  18. 2021Preserved central model for faster bidirectional compression in distributed settingscited by 21
  19. 2021Fast Federated Learning by Balancing Communication Trade-Offscited by 18
  20. 2022Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregationcited by 12
  21. 2023Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregationcited by 10
  22. 2023Gossiped and Quantized Online Multi-Kernel Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Communication Compression for Distributed Learning without Control Variatescited by 2
  2. 2025FedHQ: Hybrid Runtime Quantization for Federated Learningcited by 2
  3. 2026Joint Routing and Model Pruning for Decentralized Federated Learning in Bandwidth-Constrained Multi-Hop Wireless Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Efficient Asynchronous Federated Learning with Sparsification and Quantization
  2. 2024FedFQ: Federated Learning with Fine-Grained Quantization
  3. 2022Wireless Quantized Federated Learning: A Joint Computation and Communication Design
  4. 2026Quantization in Federated Learning: Methods, Challenges and Future Directions
  5. 2023Federated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Sampling
  6. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning
  7. 2024Rate-Constrained Quantization for Communication-Efficient Federated Learning
  8. 2021FedDQ: Communication-Efficient Federated Learning with Descending Quantization