mlsanityData to 7 Oct 2026

FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning

arXiv 2404.02478 ↗First posted 3 Apr 2024cs.LG cs.AIcited by 27

Not on the map. Why?

Abstract (arXiv)

Standard federated learning approaches suffer when client data distributions have sufficient heterogeneity. Recent methods addressed the client data heterogeneity issue via personalized federated learning (PFL) - a class of FL algorithms aiming to personalize learned global knowledge to better suit the clients' local data distributions. Existing PFL methods usually decouple global updates in deep neural networks by performing personalization on particular layers (i.e. classifier heads) and global aggregation for the rest of the network. However, preselecting network layers for personalization may result in suboptimal storage of global knowledge. In this work, we propose FedSelect, a novel PFL algorithm inspired by the iterative subnetwork discovery procedure used for the Lottery Ticket Hypothesis. FedSelect incrementally expands subnetworks to personalize client parameters, concurrently conducting global aggregations on the remaining parameters. This approach enables the personalization of both client parameters and subnetwork structure during the training process. Finally, we show that FedSelect outperforms recent state-of-the-art PFL algorithms under challenging client data heterogeneity settings and demonstrates robustness to various real-world distributional shifts. Our code is available at https://github.com/lapisrocks/fedselect.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

31 of 45 references.

References · 31

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2016Matching Networks for One Shot Learningcited by 2,246
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2014How transferable are features in deep neural networks?cited by 1,216
  8. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  9. 2018Federated Learning with Non-IID Datacited by 1,099
  10. 2017Federated Multi-Task Learningcited by 669
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  12. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
19 more
  1. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  2. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  3. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  4. 2019Federated Learning with Personalization Layerscited by 533
  5. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  6. 2021Towards Personalized Federated Learningcited by 462
  7. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  8. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  9. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  10. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  11. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  12. 2021Personalized Federated Learning using Hypernetworkscited by 210
  13. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  14. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  15. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  16. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  17. 2022Federated Learning with Partial Model Personalizationcited by 114
  18. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  19. 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 15; 2026: 6.

27 papers.

Sort:

By area:

  1. 2024F3OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristicscited by 4
  2. 2024FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clientscited by 3
  3. 2024Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformationcited by 2
  4. 2024Exploring Selective Layer Fine-Tuning in Federated Learningcited by 2
  5. 2024Optimizing Personalized Federated Learning through Adaptive Layer-Wise Learningcited by 2
  6. 2025pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixupcited by 2
  7. 2025APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squarescited by 2
  8. 2025TAP: Two-Stage Adaptive Personalization of Multi-Task and Multi-Modal Foundation Models in Federated Learningcited by 2
  9. 2025pFedFair: Towards Optimal Group Fairness-Accuracy Trade-off in Heterogeneous Federated Learningcited by 1
  10. 2024Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shiftcited by 0
  11. 2025Personalized Federated Learning for Egocentric Video Gaze Estimation with Comprehensive Parameter Frezzingcited by 0
  12. 2025FedSCA: Federated Tuning with Similarity-guided Collaborative Aggregation for Heterogeneous Medical Image Segmentationcited by 0
15 more
  1. 2025Federated Neural Architecture Search with Model-Agnostic Meta Learningcited by 0
  2. 2025Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsitycited by 0
  3. 2025FedHiP: Heterogeneity-Invariant Personalized Federated Learning Through Closed-Form Solutionscited by 0
  4. 2025Onboarding Without Forgetting: Hypernetwork Personalization with Data-Free Replay for Personalized Federated Learningcited by 0
  5. 2025Preserving Domain Generalization in Fine-Tuning via Joint Parameter Selectioncited by 0
  6. 2025Towards Privacy-Preserving and Heterogeneity-aware Split Federated Learning via Probabilistic Maskingcited by 0
  7. 2025Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participationcited by 0
  8. 2025CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networkscited by 0
  9. 2025A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learningcited by 0
  10. 2026Geometric Knowledge-Assisted Federated Dual Knowledge Distillation Approach Towards Remote Sensing Satellite Imagerycited by 0
  11. 2026HEART-PFL: Stable Personalized Federated Learning under Heterogeneity with Hierarchical Directional Alignment and Adversarial Knowledge Transfercited by 0
  12. 2026FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decouplingcited by 0
  13. 2026COSMOS: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communicationcited by 0
  14. 2026Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learningcited by 0
  15. 2026FedCKA: Representation-Guided Layer Personalization for Federated 3D Perception Across Driving Domainscited by 0