mlsanityData to 14 Oct 2026

Dynamic Heterogeneous Federated Learning with Multi-Level Prototypes

arXiv 2312.09881 ↗First posted 15 Dec 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Federated learning shows promise as a privacy-preserving collaborative learning technique. Existing heterogeneous federated learning mainly focuses on skewing the label distribution across clients. However, most approaches suffer from catastrophic forgetting and concept drift, mainly when the global distribution of all classes is extremely unbalanced and the data distribution of the client dynamically evolves over time. In this paper, we study the new task, i.e., Dynamic Heterogeneous Federated Learning (DHFL), which addresses the practical scenario where heterogeneous data distributions exist among different clients and dynamic tasks within the client. Accordingly, we propose a novel federated learning framework named Federated Multi-Level Prototypes (FedMLP) and design federated multi-level regularizations. To mitigate concept drift, we construct prototypes and semantic prototypes to provide fruitful generalization knowledge and ensure the continuity of prototype spaces. To maintain the model stability and consistency of convergence, three regularizations are introduced as training losses, i.e., prototype-based regularization, semantic prototype-based regularization, and federated inter-task regularization. Extensive experiments show that the proposed method achieves state-of-the-art performance in various settings.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

30 of 44 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2015Fast R-CNNcited by 2,933
  5. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  6. 2018LEAF: A Benchmark for Federated Settingscited by 792
  7. 2021Model-Contrastive Federated Learningcited by 766
  8. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  10. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  11. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  12. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
18 more
  1. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  2. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  3. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  4. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  5. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  6. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  7. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 161
  8. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 113
  9. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  10. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 81
  11. 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
  12. 2021Cluster-driven Graph Federated Learning over Multiple Domainscited by 44
  13. 2021FLOP: Federated Learning on Medical Datasets using Partial Networkscited by 42
  14. 2023Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learningcited by 40
  15. 2022FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillationcited by 19
  16. 2022Clustered Vehicular Federated Learning: Process and Optimizationcited by 15
  17. 2022Fast Deep Autoencoder for Federated learningcited by 10
  18. 2021GRP-FED: Addressing Client Imbalance in Federated Learning via Global-Regularized Personalizationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 1.

7 papers.

  1. 2025Personalized Federated Learning via Dual-Prompt Optimization and Cross Fusioncited by 3
  2. 2025PPFPL: Cross-silo Privacy-preserving Federated Prototype Learning Against Data Poisoning Attackscited by 1
  3. 2025Personalizing Federated Learning for Hierarchical Edge Networks with Non-IID Datacited by 1
  4. 2024Personalized Additive Modeling for Multi-level Federated Learningcited by 0
  5. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  6. 2025Communication-Efficient Neural Tangent Kernels for Heterogeneous Decentralized Federated Learningcited by 0
  7. 2026Data-Driven Pinball-Loss Selection for Vertically Distributed Elastic-Net SVMscited by 0