mlsanityData to 7 Oct 2026

FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity

arXiv 2404.09816 ↗First posted 15 Apr 2024cs.LG cs.CRcited by 13

Not on the map. Why?

Abstract (arXiv)

The interest in federated learning has surged in recent research due to its unique ability to train a global model using privacy-secured information held locally on each client. This paper pays particular attention to the issue of client-side model heterogeneity, a pervasive challenge in the practical implementation of FL that escalates its complexity. Assuming a scenario where each client possesses varied memory storage, processing capabilities and network bandwidth - a phenomenon referred to as system heterogeneity - there is a pressing need to customize a unique model for each client. In response to this, we present an effective and adaptable federated framework FedP3, representing Federated Personalized and Privacy-friendly network Pruning, tailored for model heterogeneity scenarios. Our proposed methodology can incorporate and adapt well-established techniques to its specific instances. We offer a theoretical interpretation of FedP3 and its locally differential-private variant, DP-FedP3, and theoretically validate their efficiencies.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 57 of its 72 references that are in the map. First posted on arXiv 15 Apr 2024.

Builds on

57 of 72 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2016Deep Learning with Differential Privacycited by 2,682
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  12. 2018Federated Learning with Non-IID Datacited by 1,099
38 more
  1. 2019Deep Leakage from Gradientscited by 1,004
  2. 2021Model-Contrastive Federated Learningcited by 762
  3. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  5. 2017Federated Multi-Task Learningcited by 669
  6. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  7. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  8. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  9. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  10. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  11. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  12. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  13. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  14. 2020Better Theory for SGD in the Nonconvex Worldcited by 156
  15. 2020Private Stochastic Convex Optimization: Optimal Rates in Linear Timecited by 152
  16. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  17. 2018Differentially Private Empirical Risk Minimization Revisited: Faster and More Generalcited by 127
  18. 2019Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Modelcited by 116
  19. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  20. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  21. 2020Local Differential Privacy based Federated Learning for Internet of Thingscited by 74
  22. 2018A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updatescited by 47
  23. 2022A Survey on Heterogeneous Federated Learningcited by 42
  24. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  25. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  26. 2021Permutation Compressors for Provably Faster Distributed Nonconvex Optimizationcited by 32
  27. 2014Communication-Efficient Distributed Dual Coordinate Ascentcited by 30
  28. 2020Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Searchcited by 30
  29. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  30. 2021Smoothness Matrices Beat Smoothness Constants: Better Communication Compression Techniques for Distributed Optimizationcited by 21
  31. 2019Distributed Learning of Deep Neural Networks using Independent Subnet Trainingcited by 20
  32. 2015Adding vs. Averaging in Distributed Primal-Dual Optimizationcited by 19
  33. 2020Private and Communication-Efficient Edge Learning: A Sparse Differential Gaussian-Masking Distributed SGD Approachcited by 18
  34. 2021ResIST: Layer-Wise Decomposition of ResNets for Distributed Trainingcited by 18
  35. 2022Quadratic models for understanding catapult dynamics of neural networkscited by 17
  36. 2021Super-Acceleration with Cyclical Step-sizescited by 14
  37. 2020Privacy-preserving Learning via Deep Net Pruningcited by 12
  38. 2022Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropoutcited by 12

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 1; 2026: 5.

13 papers.

Sort:

By area:

  1. 2024FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extractioncited by 21
  2. 2024Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Modelscited by 16
  3. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  4. 2024SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overheadcited by 9
  5. 2024FedSpaLLM: Federated Pruning of Large Language Modelscited by 3
  6. 2024Learnable Sparse Customization in Heterogeneous Edge Computingcited by 2
  7. 2025Symmetric Pruning of Large Language Modelscited by 2
  8. 2024Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learningcited by 1
  9. 2026SubFLOT: Submodel Extraction for Efficient and Personalized Federated Learning via Optimal Transportcited by 1
  10. 2026Byzantine-Robust Optimization under (L0, L1)-Smoothnesscited by 0
  11. 2026Representation-Aligned Multi-Scale Personalization for Federated Learningcited by 0
  12. 2026FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learningcited by 0
1 more
  1. 2026FedSAP: Federated Learning with Structured Adaptive Partitioning for Multi-Domain Heterogeneous Edge Devicescited by 0