FedORGP: Guiding Heterogeneous Federated Learning with Orthogonality Regularization on Global Prototypes
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) has emerged as an essential framework for distributed machine learning, especially with its potential for privacy-preserving data processing. However, existing FL frameworks struggle to address statistical and model heterogeneity, which severely impacts model performance. While Heterogeneous Federated Learning (HtFL) introduces prototype-based strategies to address the challenges, current approaches face limitations in achieving optimal separation of prototypes. This paper presents FedORGP, a novel HtFL algorithm designed to improve global prototype separation through orthogonality regularization, which not only encourages intra-class prototype similarity but also significantly expands the inter-class angular separation. With the guidance of the global prototype, each client keeps its embeddings aligned with the corresponding prototype in the feature space, promoting directional independence that integrates seamlessly with the cross-entropy (CE) loss. We provide theoretical proof of FedORGP's convergence under non-convex conditions. Extensive experiments demonstrate that FedORGP outperforms seven state-of-the-art baselines, achieving up to 10.12% accuracy improvement in scenarios where statistical and model heterogeneity coexist.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 22 Feb 2025.
Builds on
33 of 44 references.
References · 33
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2014Going Deeper with Convolutionscited by 4,241
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
21 more
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
- 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
- 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
- 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
- 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
- 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
- 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
- 2022Constrained Few-shot Class-incremental Learningcited by 94
- 2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 67
- 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
- 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
- 2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
- 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
- 2021Orthogonal Projection Losscited by 44
- 2024Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domainscited by 28
- 2023Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learningcited by 21
- 2023Model Barrier: A Compact Un-Transferable Isolation Domain for Model Intellectual Property Protectioncited by 14
- 2023FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learningcited by 14
- 2023FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learningcited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.