mlsanityData to 7 Oct 2026

Federated Progressive Sparsification (Purge, Merge, Tune)+

arXiv 2204.12430 ↗First posted 26 Apr 2022cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

To improve federated training of neural networks, we develop FedSparsify, a sparsification strategy based on progressive weight magnitude pruning. Our method has several benefits. First, since the size of the network becomes increasingly smaller, computation and communication costs during training are reduced. Second, the models are incrementally constrained to a smaller set of parameters, which facilitates alignment/merging of the local models and improved learning performance at high sparsification rates. Third, the final sparsified model is significantly smaller, which improves inference efficiency and optimizes operations latency during encrypted communication. We show experimentally that FedSparsify learns a subnetwork of both high sparsity and learning performance. Our sparse models can reach a tenth of the size of the original model with the same or better accuracy compared to existing pruning and nonpruning baselines.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 40 of its 58 references that are in the map. First posted on arXiv 26 Apr 2022.

Builds on

40 of 58 references.

References · 40

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  10. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  11. 2018Federated Learning with Non-IID Datacited by 1,099
  12. 2020Adaptive Federated Optimizationcited by 1,036
28 more
  1. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  2. 2019Towards Federated Learning at Scale: System Designcited by 826
  3. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  4. 2020The Future of Digital Health with Federated Learningcited by 625
  5. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  6. 2018Rethinking the Value of Network Pruningcited by 595
  7. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  8. 2019The State of Sparsity in Deep Neural Networkscited by 471
  9. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  10. 2019Federated Machine Learning: Concept and Applicationscited by 409
  11. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  12. 2021A Field Guide to Federated Optimizationcited by 312
  13. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  14. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  15. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  16. 2020Dynamic Model Pruning with Feedbackcited by 125
  17. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 122
  18. 2017Exploring Sparsity in Recurrent Neural Networkscited by 75
  19. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  20. 2020Ternary Compression for Communication-Efficient Federated Learningcited by 57
  21. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  22. 2021How Well Do Sparse Imagenet Models Transfer?cited by 50
  23. 2021SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsificationcited by 46
  24. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 32
  25. 2021FEDZIP: A Compression Framework for Communication-Efficient Federated Learningcited by 27
  26. 2020Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimizationcited by 25
  27. 2020Federated Neural Architecture Searchcited by 10
  28. 2021Sparsified Secure Aggregation for Privacy-Preserving Federated Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 4; 2025: 1.

8 papers.

  1. 2024Federated LoRA with Sparse Communicationcited by 26
  2. 2023Sparse Model Soups: A Recipe for Improved Pruning via Model Averagingcited by 20
  3. 2024SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overheadcited by 9
  4. 2025The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learningcited by 5
  5. 2024Communication-Efficient Federated Learning through Adaptive Weight Clustering and Server-Side Distillationcited by 4
  6. 2022Towards Sparsified Federated Neuroimaging Models via Weight Pruningcited by 3
  7. 2023FedCode: Communication-Efficient Federated Learning via Transferring Codebookscited by 3
  8. 2024Sparse-ProxSkip: Accelerated Sparse-to-Sparse Training in Federated Learningcited by 0