mlsanityData to 7 Oct 2026

Efficient Personalized Federated Learning via Sparse Model-Adaptation

arXiv 2305.02776 ↗First posted 4 May 2023cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) aims to train machine learning models for multiple clients without sharing their own private data. Due to the heterogeneity of clients' local data distribution, recent studies explore the personalized FL that learns and deploys distinct local models with the help of auxiliary global models. However, the clients can be heterogeneous in terms of not only local data distribution, but also their computation and communication resources. The capacity and efficiency of personalized models are restricted by the lowest-resource clients, leading to sub-optimal performance and limited practicality of personalized FL. To overcome these challenges, we propose a novel approach named pFedGate for efficient personalized FL by adaptively and efficiently learning sparse local models. With a lightweight trainable gating layer, pFedGate enables clients to reach their full potential in model capacity by generating different sparse models accounting for both the heterogeneous data distributions and resource constraints. Meanwhile, the computation and communication efficiency are both improved thanks to the adaptability between the model sparsity and clients' resources. Further, we theoretically show that the proposed pFedGate has superior complexity with guaranteed convergence and generalization error. Extensive experiments show that pFedGate achieves superior global accuracy, individual accuracy and efficiency simultaneously over state-of-the-art methods. We also demonstrate that pFedGate performs better than competitors in the novel clients participation and partial clients participation scenarios, and can learn meaningful sparse local models adapted to different data distributions.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 56 of its 82 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

56 of 82 references.

References · 56

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2017Federated Multi-Task Learningcited by 669
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  12. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
38 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  2. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  3. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  4. 2021Towards Personalized Federated Learningcited by 462
  5. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  6. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  7. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  8. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  9. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  10. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  11. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  12. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  13. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  14. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  15. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  16. 2021Personalized Federated Learning using Hypernetworkscited by 210
  17. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  18. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  19. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  20. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  21. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  22. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  23. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  24. 2020TiFL: A Tier-based Federated Learning Systemcited by 95
  25. 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
  26. 2019Variational Federated Multi-Task Learningcited by 69
  27. 2021Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modelingcited by 69
  28. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  29. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  30. 2022FederatedScope: A Flexible Federated Learning Platform for Heterogeneitycited by 53
  31. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  32. 2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
  33. 2022FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learningcited by 43
  34. 2021FLOP: Federated Learning on Medical Datasets using Partial Networkscited by 42
  35. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  36. 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 34
  37. 2018Differentiable Learning-to-Normalize via Switchable Normalizationcited by 33
  38. 2021QuPeD: Quantized Personalization via Distillation with Applications to Federated Learningcited by 32

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 17; 2025: 6; 2026: 6.

38 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93
  3. 2023Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytescited by 55
  4. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
  5. 2023Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaborationcited by 23
  6. 2024Towards Robust and Efficient Federated Low-Rank Adaptation with Heterogeneous Clientscited by 22
  7. 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
  8. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learningcited by 21
  9. 2024FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extractioncited by 21
  10. 2024On the Convergence of Zeroth-Order Federated Tuning for Large Language Modelscited by 18
  11. 2024Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decompositioncited by 15
  12. 2023ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learningcited by 9
26 more
  1. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9
  2. 2024Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approachcited by 8
  3. 2024Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelscited by 6
  4. 2023Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learningcited by 5
  5. 2024F3OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristicscited by 4
  6. 2024Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Schemecited by 3
  7. 2024Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clientscited by 3
  8. 2024Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformationcited by 2
  9. 2024Exploring Selective Layer Fine-Tuning in Federated Learningcited by 2
  10. 2023Federated Learning in IoT: a Survey from a Resource-Constrained Perspectivecited by 1
  11. 2024Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guaranteescited by 1
  12. 2025FedPaI: Achieving Extreme Sparsity in Federated Learning via Pruning at Initializationcited by 1
  13. 2025Hypernetworks for Model-Heterogeneous Personalized Federated Learningcited by 1
  14. 2023Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Datacited by 0
  15. 2024Selective Knowledge Sharing for Personalized Federated Learning Under Capacity Heterogeneitycited by 0
  16. 2024Recovering Global Data Distribution Locally in Federated Learningcited by 0
  17. 2025Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensionscited by 0
  18. 2025Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsitycited by 0
  19. 2025From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0cited by 0
  20. 2025DART: A Server-side Plug-in for Resource-efficient Robust Federated Learningcited by 0
  21. 2026Federated Personalization of Early-Exit Networkscited by 0
  22. 2026Representation-Aligned Multi-Scale Personalization for Federated Learningcited by 0
  23. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0
  24. 2026Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learningcited by 0
  25. 2026FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learningcited by 0
  26. 2026FedSAP: Federated Learning with Structured Adaptive Partitioning for Multi-Domain Heterogeneous Edge Devicescited by 0