Hypernetworks for Model-Heterogeneous Personalized Federated Learning
Not on the map. Why?
Abstract (arXiv)
Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning strategies, which can limit their practicality and scalability. In this paper, we revisit hypernetwork-based methods and leverage their strong generalization capabilities to design a simple yet effective framework for heterogeneous personalized federated learning. Specifically, we propose MH-pFedHN, which leverages a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model. To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads. Furthermore, we further propose MH-pFedHNGD, which integrates an optional lightweight global model to improve generalization. Our framework does not rely on external datasets and does not require disclosure of client model architectures, thereby offering enhanced privacy and flexibility. Extensive experiments on multiple benchmarks and model settings demonstrate that our approach achieves competitive accuracy, strong generalization, and serves as a robust baseline for future research in model-heterogeneous personalized federated learning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 53 of its 77 references that are in the map. First posted on arXiv 30 Jul 2025.
Builds on
53 of 77 references.
References · 53
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2022Generative Adversarial Networkscited by 5,627
- 2016Wide Residual Networkscited by 3,070
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
- 2016HyperNetworkscited by 1,027
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2017Federated Multi-Task Learningcited by 672
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
38 more
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
- 2019Federated Learning with Personalization Layerscited by 541
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
- 2020iDLG: Improved Deep Leakage from Gradientscited by 382
- 2020Adaptive Personalized Federated Learningcited by 358
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
- 2019Continual learning with hypernetworkscited by 273
- 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 243
- 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
- 2021Personalized Federated Learning using Hypernetworkscited by 212
- 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
- 2021Federated Learning for Smart Healthcare: A Surveycited by 184
- 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
- 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
- 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
- 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
- 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
- 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
- 2020TiFL: A Tier-based Federated Learning Systemcited by 96
- 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 96
- 2020HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentationcited by 66
- 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
- 2022Federated Unlearning for On-Device Recommendationcited by 43
- 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
- 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
- 2020Communication-Efficient Federated Distillationcited by 28
- 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
- 2022HyperShot: Few-Shot Learning by Kernel HyperNetworkscited by 24
- 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
- 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
- 2022Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillationcited by 22
- 2023FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregationcited by 22
- 2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
- 2022Federated Learning with Heterogeneous Architectures using Graph HyperNetworkscited by 19
- 2023Recurrent Hypernetworks are Surprisingly Strong in Meta-RLcited by 16
- 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.