mlsanityData to 7 Oct 2026

Sparse Federated Learning with Hierarchical Personalized Models

arXiv 2203.13517 ↗First posted 25 Mar 2022cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) can achieve privacy-safe and reliable collaborative training without collecting users' private data. Its excellent privacy security potential promotes a wide range of FL applications in Internet-of-Things (IoT), wireless networks, mobile devices, autonomous vehicles, and cloud medical treatment. However, the FL method suffers from poor model performance on non-i.i.d. data and excessive traffic volume. To this end, we propose a personalized FL algorithm using a hierarchical proximal mapping based on the moreau envelop, named sparse federated learning with hierarchical personalized models (sFedHP), which significantly improves the global model performance facing diverse data. A continuously differentiable approximated L1-norm is also used as the sparse constraint to reduce the communication cost. Convergence analysis shows that sFedHP's convergence rate is state-of-the-art with linear speedup and the sparse constraint only reduces the convergence rate to a small extent while significantly reducing the communication cost. Experimentally, we demonstrate the benefits of sFedHP compared with the FedAvg, HierFAVG (hierarchical FedAvg), and personalized FL methods based on local customization, including FedAMP, FedProx, Per-FedAvg, pFedMe, and pFedGP.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

33 of 52 references.

References · 33

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2019Towards Federated Learning at Scale: System Designcited by 826
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  11. 2019Federated Learning with Personalization Layerscited by 533
  12. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
21 more
  1. 2019The State of Sparsity in Deep Neural Networkscited by 471
  2. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  3. 2020Adaptive Personalized Federated Learningcited by 353
  4. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  5. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  6. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  7. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  8. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  9. 2019A Joint Learning and Communications Framework for Federated Learning over Wireless Networkscited by 262
  10. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  11. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  12. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 138
  13. 2019One-Shot Federated Learningcited by 124
  14. 2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
  15. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  16. 2020Communication-Efficient Edge AI: Algorithms and Systemscited by 95
  17. 2021Personalized Federated Learning with Gaussian Processescited by 76
  18. 2016Training Sparse Neural Networkscited by 73
  19. 2015Complete Dictionary Recovery over the Sphere I: Overview and the Geometric Picturecited by 49
  20. 2019Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learningcited by 18
  21. 2021Sparse Personalized Federated Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 1; 2025: 2.

4 papers.

  1. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  2. 2024Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasetscited by 1
  3. 2025A Randomized Zeroth-Order Hierarchical Framework for Heterogeneous Federated Learningcited by 0
  4. 2025HFedATM: Hierarchical Federated Domain Generalization via Optimal Transport and Regularized Mean Aggregationcited by 0