mlsanityData to 14 Oct 2026

Heterogeneous Federated Learning with Prototype Alignment and Upscaling

arXiv 2507.04310 ↗First posted 6 Jul 2025cs.LG cs.DCcited by 0

Not on the map. Why?

Abstract (arXiv)

Heterogeneity in data distributions and model architectures remains a significant challenge in federated learning (FL). Various heterogeneous FL (HtFL) approaches have recently been proposed to address this challenge. Among them, prototype-based FL (PBFL) has emerged as a practical framework that only shares per-class mean activations from the penultimate layer. However, PBFL approaches often suffer from suboptimal prototype separation, limiting their discriminative power. We propose Prototype Normalization (ProtoNorm), a novel PBFL framework that addresses this limitation through two key components: Prototype Alignment (PA) and Prototype Upscaling (PU). The PA method draws inspiration from the Thomson problem in classical physics, optimizing global prototype configurations on a unit sphere to maximize angular separation; subsequently, the PU method increases prototype magnitudes to enhance separation in Euclidean space. Extensive evaluations on benchmark datasets show that our approach better separates prototypes and thus consistently outperforms existing HtFL approaches. Notably, since ProtoNorm inherits the communication efficiency of PBFL and the PA is performed server-side, it is particularly suitable for resource-constrained environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 45 references that are in the map. First posted on arXiv 6 Jul 2025.

Builds on

31 of 45 references.

References · 31

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  4. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 971
  5. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  6. 2017Deep Mutual Learningcited by 628
  7. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  8. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  9. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  10. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  11. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  12. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
19 more
  1. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  2. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  3. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  4. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
  5. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  6. 2018Hyperspherical Variational Auto-Encoderscited by 189
  7. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 122
  8. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  9. 2018Learning towards Minimum Hyperspherical Energycited by 75
  10. 2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 67
  11. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  12. 2017Deep Hyperspherical Learningcited by 55
  13. 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
  14. 2020Orthogonal Over-Parameterized Trainingcited by 44
  15. 2021Learning with Hyperspherical Uniformitycited by 35
  16. 2019Angular Visual Hardnesscited by 29
  17. 2019Regularizing Neural Networks via Minimizing Hyperspherical Energycited by 25
  18. 2022Hyperspherical Consistency Regularizationcited by 16
  19. 2024FedUV: Uniformity and Variance for Heterogeneous Federated Learningcited by 9

Built on by

Not cited yet.