mlsanityData to 7 Oct 2026

Personalized Federated Learning on Heterogeneous and Long-Tailed Data via Expert Collaborative Learning

arXiv 2408.02019 ↗First posted 4 Aug 2024cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Personalized Federated Learning (PFL) aims to acquire customized models for each client without disclosing raw data by leveraging the collective knowledge of distributed clients. However, the data collected in real-world scenarios is likely to follow a long-tailed distribution. For example, in the medical domain, it is more common for the number of general health notes to be much larger than those specifically relatedto certain diseases. The presence of long-tailed data can significantly degrade the performance of PFL models. Additionally, due to the diverse environments in which each client operates, data heterogeneity is also a classic challenge in federated learning. In this paper, we explore the joint problem of global long-tailed distribution and data heterogeneity in PFL and propose a method called Expert Collaborative Learning (ECL) to tackle this problem. Specifically, each client has multiple experts, and each expert has a different training subset, which ensures that each class, especially the minority classes, receives sufficient training. Multiple experts collaborate synergistically to produce the final prediction output. Without special bells and whistles, the vanilla ECL outperforms other state-of-the-art PFL methods on several benchmark datasets under different degrees of data heterogeneity and long-tailed distribution.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 4 Aug 2024.

Builds on

35 of 47 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  7. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  10. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  11. 2019Federated Learning with Personalization Layerscited by 533
  12. 2021Towards Personalized Federated Learningcited by 462
23 more
  1. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  2. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 397
  3. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  4. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  5. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  6. 2020Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classificationcited by 175
  7. 2020Disentangling Label Distribution for Long-tailed Visual Recognitioncited by 172
  8. 2019Federated Evaluation of On-device Personalizationcited by 147
  9. 2020Survey of Personalization Techniques for Federated Learningcited by 144
  10. 2020M2m: Imbalanced Classification via Major-to-minor Translationcited by 134
  11. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  12. 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
  13. 2020TiFL: A Tier-based Federated Learning Systemcited by 95
  14. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 95
  15. 2021FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentationcited by 76
  16. 2022Nested Collaborative Learning for Long-Tailed Visual Recognitioncited by 63
  17. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  18. 2020FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoringcited by 61
  19. 2022CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learningcited by 34
  20. 2023Long-Tailed Visual Recognition via Self-Heterogeneous Integration with Knowledge Excavationcited by 22
  21. 2021Feature Generation for Long-tail Classificationcited by 9
  22. 2023Personalized Federated Learning on Long-Tailed Data via Adversarial Feature Augmentationcited by 5
  23. 2022Personalized Federated Learning with Exact Stochastic Gradient Descentcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025S2FGL: Spatial Spectral Federated Graph Learningcited by 7