mlsanityData to 14 Oct 2026

Efficient Asynchronous Federated Learning with Sparsification and Quantization

arXiv 2312.15186 ↗First posted 23 Dec 2023cs.DC cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

While data is distributed in multiple edge devices, Federated Learning (FL) is attracting more and more attention to collaboratively train a machine learning model without transferring raw data. FL generally exploits a parameter server and a large number of edge devices during the whole process of the model training, while several devices are selected in each round. However, straggler devices may slow down the training process or even make the system crash during training. Meanwhile, other idle edge devices remain unused. As the bandwidth between the devices and the server is relatively low, the communication of intermediate data becomes a bottleneck. In this paper, we propose Time-Efficient Asynchronous federated learning with Sparsification and Quantization, i.e., TEASQ-Fed. TEASQ-Fed can fully exploit edge devices to asynchronously participate in the training process by actively applying for tasks. We utilize control parameters to choose an appropriate number of parallel edge devices, which simultaneously execute the training tasks. In addition, we introduce a caching mechanism and weighted averaging with respect to model staleness to further improve the accuracy. Furthermore, we propose a sparsification and quantitation approach to compress the intermediate data to accelerate the training. The experimental results reveal that TEASQ-Fed improves the accuracy (up to 16.67% higher) while accelerating the convergence of model training (up to twice faster).

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 23 Dec 2023.

Builds on

39 of 58 references.

References · 39

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  5. 2018Federated Learning with Non-IID Datacited by 1,108
  6. 2021Model-Contrastive Federated Learningcited by 766
  7. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  8. 2019Federated Machine Learning: Concept and Applicationscited by 412
  9. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  10. 2018Sparsified SGD with Memorycited by 376
  11. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  12. 2019Asynchronous Federated Optimizationcited by 267
27 more
  1. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  2. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  3. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 190
  4. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 181
  5. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 100
  6. 2021Asynchronous Federated Learning on Heterogeneous Devices: A Surveycited by 93
  7. 2020Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learningcited by 90
  8. 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
  9. 2018In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learningcited by 80
  10. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 79
  11. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 66
  12. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
  13. 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
  14. 2023FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Updatecited by 32
  15. 2021Efficient Device Scheduling with Multi-Job Federated Learningcited by 29
  16. 2022FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Servercited by 29
  17. 2019Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computingcited by 28
  18. 2020CSER: Communication-efficient SGD with Error Resetcited by 27
  19. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  20. 2022FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneitycited by 20
  21. 2022Multi-Job Intelligent Scheduling with Cross-Device Federated Learningcited by 20
  22. 2021HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environmentscited by 15
  23. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  24. 2023Distributed and Deep Vertical Federated Learning with Big Datacited by 10
  25. 2018A Distributed Collaborative Filtering Algorithm Using Multiple Data Sourcescited by 2
  26. 2023On Mask-based Image Set Desensitization with Recognition Supportcited by 2
  27. 2019Adaptive Task Allocation for Asynchronous Federated and Parallelized Mobile Edge Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 4.

9 papers.

  1. 2024Trustworthy Federated Learning: Privacy, Security, and Beyondcited by 11
  2. 2024Enhancing Trust and Privacy in Distributed Networks: A Comprehensive Survey on Blockchain-based Federated Learningcited by 9
  3. 2024Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Datacited by 6
  4. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  5. 2024Federated Learning in Chemical Engineering: A Tutorial on a Framework for Privacy-Preserving Collaboration Across Distributed Data Sourcescited by 4
  6. 2025SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignmentcited by 3
  7. 2025Efficient Federated Learning with Timely Update Disseminationcited by 2
  8. 2025Efficient Federated Learning with Heterogeneous Data and Adaptive Dropoutcited by 2
  9. 2025Adaptive Deadline and Batch Layered Synchronized Federated Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Quantized and Asynchronous Federated Learning
  2. 2023Asynchronous Federated Learning with Bidirectional Quantized Communications and Buffered Aggregation
  3. 2022Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks
  4. 2022Communication-Efficient Federated Learning With Data and Client Heterogeneity
  5. 2024Asynchronous Byzantine Federated Learning
  6. 2021Time-Correlated Sparsification for Communication-Efficient Federated Learning
  7. 2024FedFQ: Federated Learning with Fine-Grained Quantization
  8. 2022Communication-Efficient Federated Learning for Heterogeneous Edge Devices Based on Adaptive Gradient Quantization