mlsanityData to 14 Oct 2026

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

arXiv 2411.01825 ↗First posted 4 Nov 2024cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a distributed machine learning paradigm that achieves a globally robust model through decentralized computation and periodic model synthesis, primarily focusing on the global model's accuracy over aggregated datasets of all participating clients. Personalized Federated Learning (PFL) instead tailors exclusive models for each client, aiming to enhance the accuracy of clients' individual models on specific local data distributions. Despite of their wide adoption, existing FL and PFL works have yet to comprehensively address the class-imbalance issue, one of the most critical challenges within the realm of data heterogeneity in PFL and FL research. In this paper, we propose FedReMa, an efficient PFL algorithm that can tackle class-imbalance by 1) utilizing an adaptive inter-client co-learning approach to identify and harness different clients' expertise on different data classes throughout various phases of the training process, and 2) employing distinct aggregation methods for clients' feature extractors and classifiers, with the choices informed by the different roles and implications of these model components. Specifically, driven by our experimental findings on inter-client similarity dynamics, we develop critical co-learning period (CCP), wherein we introduce a module named maximum difference segmentation (MDS) to assess and manage task relevance by analyzing the similarities between clients' logits of their classifiers. Outside the CCP, we employ an additional scheme for model aggregation that utilizes historical records of each client's most relevant peers to further enhance the personalization stability. We demonstrate the superiority of our FedReMa in extensive experiments.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 4 Nov 2024.

Builds on

25 of 33 references.

References · 25

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2020Knowledge Distillation: A Surveycited by 1,603
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  5. 2018Federated Learning with Non-IID Datacited by 1,108
  6. 2021Model-Contrastive Federated Learningcited by 766
  7. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  8. 2019Federated Learning with Personalization Layerscited by 541
  9. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  10. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  11. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  12. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
13 more
  1. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
  2. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  3. 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
  4. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  5. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
  6. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  7. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  8. 2019Active Federated Learningcited by 63
  9. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  10. 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 50
  11. 2024FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuningcited by 27
  12. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  13. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixupcited by 3
  2. 2025Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learningcited by 1
  3. 2025Decentralized Rank Scheduling for Energy-Constrained Multi-Task Federated Fine-Tuning in Edge-Assisted IoV Networkscited by 0