mlsanityData to 7 Oct 2026

Sparse Personalized Federated Learning

arXiv 2107.05330 ↗First posted 12 Jul 2021cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a collaborative machine learning technique to train a global model without obtaining clients' private data. The main challenges in FL are statistical diversity among clients, limited computing capability among clients' equipments, and the excessive communication overhead between the server and clients. To address these challenges, we propose a novel sparse personalized federated learning scheme via maximizing correlation (FedMac). By incorporating an approximated L1-norm and the correlation between client models and global model into standard FL loss function, the performance on statistical diversity data is improved and the communicational and computational loads required in the network are reduced compared with non-sparse FL. Convergence analysis shows that the sparse constraints in FedMac do not affect the convergence rate of the global model, and theoretical results show that FedMac can achieve good sparse personalization, which is better than the personalized methods based on L2-norm. Experimentally, we demonstrate the benefits of this sparse personalization architecture compared with the state-of-the-art personalization methods (e.g. FedMac respectively achieves 98.95%, 99.37%, 90.90%, 89.06% and 73.52% accuracy on the MNIST, FMNIST, CIFAR-100, Synthetic and CINIC-10 datasets under non-i.i.d. variants).

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 36 of its 61 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

36 of 61 references.

References · 36

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020The Future of Digital Health with Federated Learningcited by 625
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  12. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
24 more
  1. 2019Federated Learning with Personalization Layerscited by 533
  2. 2019The State of Sparsity in Deep Neural Networkscited by 471
  3. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 430
  4. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  5. 2020Adaptive Personalized Federated Learningcited by 353
  6. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  7. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  8. 2017Deep Models Under the GAN: Information Leakage from Collaborative Deep Learningcited by 326
  9. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  10. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  11. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  12. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  13. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  14. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  15. 2019One-Shot Federated Learningcited by 124
  16. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  17. 2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 103
  18. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 99
  19. 2019Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systemscited by 77
  20. 2016Training Sparse Neural Networkscited by 73
  21. 2020Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Frameworkcited by 59
  22. 2015Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picturecited by 49
  23. 2019Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learningcited by 18
  24. 2020Directional Pruning of Deep Neural Networkscited by 16

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 2; 2025: 1.

4 papers.

  1. 2024SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overheadcited by 9
  2. 2022Sparse Federated Learning with Hierarchical Personalized Modelscited by 4
  3. 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
  4. 2025From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0cited by 0