mlsanityData to 14 Oct 2026

SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communications

arXiv 2204.02321 ↗First posted 5 Apr 2022cs.DC cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) enables edge devices to collaboratively learn a model in a distributed fashion. Many existing researches have focused on improving communication efficiency of high-dimensional models and addressing bias caused by local updates. However, most of FL algorithms are either based on reliable communications or assume fixed and known unreliability characteristics. In practice, networks could suffer from dynamic channel conditions and non-deterministic disruptions, with time-varying and unknown characteristics. To this end, in this paper we propose a sparsity enabled FL framework with both communication efficiency and bias reduction, termed as SAFARI. It makes novel use of a similarity among client models to rectify and compensate for bias that is resulted from unreliable communications. More precisely, sparse learning is implemented on local clients to mitigate communication overhead, while to cope with unreliable communications, a similarity-based compensation method is proposed to provide surrogates for missing model updates. We analyze SAFARI under bounded dissimilarity and with respect to sparse models. It is demonstrated that SAFARI under unreliable communications is guaranteed to converge at the same rate as the standard FedAvg with perfect communications. Implementations and evaluations on CIFAR-10 dataset validate the effectiveness of SAFARI by showing that it can achieve the same convergence speed and accuracy as FedAvg with perfect communications, with up to 80% of the model weights being pruned and a high percentage of client updates missing in each round.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

30 of 42 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2018Federated Learning with Non-IID Datacited by 1,108
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  9. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  10. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  11. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  12. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
18 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  2. 2018Sparsified SGD with Memorycited by 376
  3. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  4. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  5. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
  6. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  7. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  8. 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 213
  9. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 201
  10. 2019Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networkscited by 160
  11. 2020Is Local SGD Better than Minibatch SGD?cited by 151
  12. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  13. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 63
  14. 2016AIDE: Fast and Communication Efficient Distributed Optimizationcited by 43
  15. 2021Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learningcited by 31
  16. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 31
  17. 2020Federated Learning and Wireless Communicationscited by 27
  18. 2021Effective Model Sparsification by Scheduled Grow-and-Prune Methodscited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 3; 2026: 1.

11 papers.

  1. 2024Advances and Open Challenges in Federated Foundation Modelscited by 36
  2. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
  3. 2025Unified Analysis of Decentralized Gradient Descent: a Contraction Mapping Frameworkcited by 10
  4. 2024FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clusteringcited by 5
  5. 2023Keep It Simple: Fault Tolerance Evaluation of Federated Learning with Unreliable Clientscited by 2
  6. 2023Inclusive Data Representation in Federated Learning: A Novel Approach Integrating Textual and Visual Promptcited by 1
  7. 2024Adaptive Quantization Resolution and Power Control for Federated Learning over Cell-free Networkscited by 1
  8. 2024Accelerating Energy-Efficient Federated Learning in Cell-Free Networks with Adaptive Quantizationcited by 1
  9. 2025A Two-Timescale Approach for Wireless Federated Learning with Parameter Freezing and Power Controlcited by 0
  10. 2025Traversal Learning: A Lossless And Efficient Distributed Learning Frameworkcited by 0
  11. 2026Prioritizing Gradient Sign Over Modulus: An Importance-Aware Framework for Wireless Federated Learningcited by 0