mlsanityData to 7 Oct 2026

FedGH: Heterogeneous Federated Learning with Generalized Global Header

arXiv 2303.13137 ↗First posted 23 Mar 2023cs.LG cs.DCcited by 56

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) is an emerging machine learning paradigm that allows multiple parties to train a shared model collaboratively in a privacy-preserving manner. Existing horizontal FL methods generally assume that the FL server and clients hold the same model structure. However, due to system heterogeneity and the need for personalization, enabling clients to hold models with diverse structures has become an important direction. Existing model-heterogeneous FL approaches often require publicly available datasets and incur high communication and/or computational costs, which limit their performances. To address these limitations, we propose a simple but effective Federated Global prediction Header (FedGH) approach. It is a communication and computation-efficient model-heterogeneous FL framework which trains a shared generalized global prediction header with representations extracted by heterogeneous extractors for clients' models at the FL server. The trained generalized global prediction header learns from different clients. The acquired global knowledge is then transferred to clients to substitute each client's local prediction header. We derive the non-convex convergence rate of FedGH. Extensive experiments on two real-world datasets demonstrate that FedGH achieves significantly more advantageous performance in both model-homogeneous and -heterogeneous FL scenarios compared to seven state-of-the-art personalized FL models, beating the best-performing baseline by up to 8.87% (for model-homogeneous FL) and 1.83% (for model-heterogeneous FL) in terms of average test accuracy, while saving up to 85.53% of communication overhead.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

32 of 49 references.

References · 32

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  3. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  4. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  5. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  6. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  7. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  8. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  9. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  10. 2021Personalized Federated Learning using Hypernetworkscited by 210
  11. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  12. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
20 more
  1. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  2. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  3. 2022Federated Learning with Partial Model Personalizationcited by 114
  4. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  5. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  6. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  7. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 76
  8. 2021GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learningcited by 73
  9. 2020Federated Mutual Learningcited by 70
  10. 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
  11. 2021Towards Fairness-Aware Federated Learningcited by 57
  12. 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
  13. 2021Fed2: Feature-Aligned Federated Learningcited by 42
  14. 2019Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Datacited by 39
  15. 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
  16. 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
  17. 2021FedMatch: Federated Learning Over Heterogeneous Question Answering Datacited by 23
  18. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  19. 2020Cooperative Learning via Federated Distillation over Fading Channelscited by 16
  20. 2022Completely Heterogeneous Federated Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 16; 2025: 21; 2026: 12.

56 papers.

Sort:

By area:

  1. 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
  2. 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 38
  3. 2024Advances and Open Challenges in Federated Foundation Modelscited by 33
  4. 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
  5. 2024pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learningcited by 21
  6. 2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
  7. 2024An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learningcited by 17
  8. 2023PFLlib: A Beginner-Friendly and Comprehensive Personalized Federated Learning Library and Benchmarkcited by 16
  9. 2024Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decompositioncited by 15
  10. 2023FedBEVT: Federated Learning Bird's Eye View Perception Transformer in Road Traffic Systemscited by 14
  11. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  12. 2023FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learningcited by 10
38 more
  1. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
  2. 2024DapperFL: Domain Adaptive Federated Learning with Model Fusion Pruning for Edge Devicescited by 9
  3. 2025Class-wise Balancing Data Replay for Federated Class-Incremental Learningcited by 9
  4. 2025PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learningcited by 8
  5. 2025HtFLlib: A Comprehensive Heterogeneous Federated Learning Library and Benchmarkcited by 7
  6. 2024Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuningcited by 6
  7. 2025Federated Out-of-Distribution Generalization: A Causal Augmentation Viewcited by 4
  8. 2025FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Modelscited by 4
  9. 2025Federated Deconfounding and Debiasing Learning for Out-of-Distribution Generalizationcited by 4
  10. 2024pFedAFM: Adaptive Feature Mixture for Batch-Level Personalization in Heterogeneous Federated Learningcited by 3
  11. 2024FedCoSR: Personalized Federated Learning with Contrastive Shareable Representations for Label Heterogeneity in Non-IID Datacited by 3
  12. 2024FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillationcited by 3
  13. 2024Asymmetrical Reciprocity-based Federated Learning for Resolving Disparities in Medical Diagnosiscited by 2
  14. 2025Enhancing Visual Representation with Textual Semantics: Textual Semantics-Powered Prototypes for Heterogeneous Federated Learningcited by 2
  15. 2026Tackling Resource-Constrained and Data-Heterogeneity in Federated Learning with Double-Weight Sparse Packcited by 2
  16. 2024FedPAE: Peer-Adaptive Ensemble Learning for Asynchronous and Model-Heterogeneous Federated Learningcited by 1
  17. 2025Beyond Model Scale Limits: End-Edge-Cloud Federated Learning with Self-Rectified Knowledge Agglomerationcited by 1
  18. 2025FedRIR: Rethinking Information Representation in Federated Learningcited by 1
  19. 2025FedORGP: Guiding Heterogeneous Federated Learning with Orthogonality Regularization on Global Prototypescited by 1
  20. 2025Robust Federated Learning with Global Sensitivity Estimation for Financial Risk Managementcited by 1
  21. 2025Conditioning on Local Statistics for Scalable Heterogeneous Federated Learningcited by 1
  22. 2025Generalized and Personalized Federated Learning with Black-Box Foundation Models via Orthogonal Transformationscited by 1
  23. 2025Hypernetworks for Model-Heterogeneous Personalized Federated Learningcited by 1
  24. 2025FedRE: A Representation Entanglement Framework for Model-Heterogeneous Federated Learningcited by 1
  25. 2023FedAgg: Adaptive Federated Learning with Aggregated Gradientscited by 0
  26. 2024Adaptive Guidance for Local Training in Heterogeneous Federated Learningcited by 0
  27. 2024GeFL: Model-Agnostic Federated Learning with Generative Modelscited by 0
  28. 2025Knowledge-Driven Federated Graph Learning on Model Heterogeneitycited by 0
  29. 2025FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Datacited by 0
  30. 2025MoCFL: Mobile Cluster Federated Learning Framework for Highly Dynamic Networkcited by 0
  31. 2025Heterogeneous Federated Learning with Prototype Alignment and Upscalingcited by 0
  32. 2025DART: A Server-side Plug-in for Resource-efficient Robust Federated Learningcited by 0
  33. 2025DPFNAS: Differential Privacy-Enhanced Federated Neural Architecture Search for 6G Edge Intelligencecited by 0
  34. 2026RefProtoFL: Communication-Efficient Federated Learning via External-Referenced Prototype Alignmentcited by 0
  35. 2026SDFed: Bridging Local Global Discrepancy via Subspace Refinement and Divergence Control in Federated Prompt Learningcited by 0
  36. 2026Social Hippocampus Memory Learningcited by 0
  37. 2026FedDES: Graph-Based Dynamic Ensemble Selection for Personalized Federated Learningcited by 0
  38. 2026From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneitycited by 0

50 most cited of 56 All 56 (JSON)