mlsanityData to 7 Oct 2026

SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead

arXiv 2406.00431 ↗First posted 1 Jun 2024cs.LG cs.AI cs.DCcited by 9

Not on the map. Why?

Abstract (arXiv)

The large communication and computation overhead of federated learning (FL) is one of the main challenges facing its practical deployment over resource-constrained clients and systems. In this work, SpaFL: a communication-efficient FL framework is proposed to optimize sparse model structures with low computational overhead. In SpaFL, a trainable threshold is defined for each filter/neuron to prune its all connected parameters, thereby leading to structured sparsity. To optimize the pruning process itself, only thresholds are communicated between a server and clients instead of parameters, thereby learning how to prune. Further, global thresholds are used to update model parameters by extracting aggregated parameter importance. The generalization bound of SpaFL is also derived, thereby proving key insights on the relation between sparsity and performance. Experimental results show that SpaFL improves accuracy while requiring much less communication and computing resources compared to sparse baselines. The code is available at https://github.com/news-vt/SpaFL_NeruIPS_2024

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 1 Jun 2024.

Builds on

36 of 46 references.

References · 36

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2015Automatic differentiation in machine learning: a surveycited by 846
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  8. 2019Importance Estimation for Neural Network Pruningcited by 531
  9. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  10. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  11. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  12. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
24 more
  1. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
  2. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  3. 2019A Survey on Distributed Machine Learningcited by 187
  4. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  5. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  6. 2020Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscited by 66
  7. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  8. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  9. 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
  10. 2021Efficient Neural Network Training via Forward and Backward Propagation Sparsificationcited by 30
  11. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 30
  12. 2020Communication-Efficient Distributed Deep Learning: A Comprehensive Surveycited by 25
  13. 2023Dynamic Sparsity Is Channel-Level Sparsity Learnercited by 17
  14. 2021Masked Training of Neural Networks with Partial Gradientscited by 15
  15. 2020Tighter Generalization Bounds for Iterative Differentially Private Learning Algorithmscited by 14
  16. 2021SPIDER: Searching Personalized Neural Architecture for Federated Learningcited by 13
  17. 2024FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneitycited by 13
  18. 2022Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edgecited by 12
  19. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  20. 2022Partial Model Averaging in Federated Learning: Performance Guarantees and Benefitscited by 8
  21. 2022Federated Progressive Sparsification (Purge, Merge, Tune)+cited by 8
  22. 2022SlimFL: Federated Learning with Superposition Coding over Slimmable Neural Networkscited by 6
  23. 2021Sparse Personalized Federated Learningcited by 4
  24. 2022Supernet Training for Federated Image Classification under System Heterogeneitycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 6.

9 papers.

  1. 2025FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learningcited by 2
  2. 2025Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learningcited by 1
  3. 2025Federated CLIP for Resource-Efficient Heterogeneous Medical Image Classificationcited by 1
  4. 2026SA-PEF: Step-Ahead Partial Error Feedback for Efficient Federated Learningcited by 0
  5. 2026GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Taskcited by 0
  6. 2026Federated Learning of Binary Neural Networks: Enabling Low-Cost Inferencecited by 0
  7. 2026QuantFL: Sustainable Federated Learning for Edge IoT via Pre-Trained Model Quantisationcited by 0
  8. 2026On the Push-Based Asynchronous Federated Learning: A Bias-Correction Aggregation Approachcited by 0
  9. 2026Communication-Efficient Personalized Federated Learning via Layer-Wise Multi-Threshold Random Sketchingcited by 0