Data to 7 Oct 2026

Personalized Federated Learning using Hypernetworks

arXiv 2103.04628 ↗First posted 8 Mar 2021cs.LGcited by 210

Not on the map. Why?

Abstract (arXiv)

Personalized federated learning is tasked with training machine learning models for multiple clients, each with its own data distribution. The goal is to train personalized models in a collaborative way while accounting for data disparities across clients and reducing communication costs. We propose a novel approach to this problem using hypernetworks, termed pFedHN for personalized Federated HyperNetworks. In this approach, a central hypernetwork model is trained to generate a set of models, one model for each client. This architecture provides effective parameter sharing across clients, while maintaining the capacity to generate unique and diverse personal models. Furthermore, since hypernetwork parameters are never transmitted, this approach decouples the communication cost from the trainable model size. We test pFedHN empirically in several personalized federated learning challenges and find that it outperforms previous methods. Finally, since hypernetworks share information across clients we show that pFedHN can generalize better to new clients whose distributions differ from any client observed during training.

Key facts. Cited by 210 papers in the map (data through 7 Oct 2026). Builds on 44 of its 64 references that are in the map. First posted on arXiv 8 Mar 2021.

Builds on

44 of 64 references.

References · 44

2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
2019Advances and Open Problems in Federated Learningcited by 3,021
2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
2018Federated Learning with Non-IID Datacited by 1,099
2018On First-Order Meta-Learning Algorithmscited by 1,003
2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
2017Federated Multi-Task Learningcited by 669
2020Personalized Federated Learning with Moreau Envelopescited by 612
2018Local SGD Converges Fast and Communicates Littlecited by 595
2019Federated Learning with Personalization Layerscited by 533

32 more

2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
2020Adaptive Personalized Federated Learningcited by 353
2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
2019Continual learning with hypernetworkscited by 267
2020Federated Learning of a Mixture of Global and Local Modelscited by 208
2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
2020Personalized Federated Learning with First Order Model Optimizationcited by 201
2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
2020Survey of Personalization Techniques for Federated Learningcited by 144
2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
2020Learning the Pareto Front with Hypernetworkscited by 122
2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
2019Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functionscited by 104
2017On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimizationcited by 91
2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
2018Stochastic Hyperparameter Optimization through Hypernetworkscited by 72
2019Hypernetwork functional image representationcited by 65
2020Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge based Frameworkcited by 59
2019Hyper-Graph-Network Decoders for Block Codescited by 42
2019Federated Heavy Hitters Discovery with Differential Privacycited by 32
2019Alpha MAML: Adaptive Model-Agnostic Meta-Learningcited by 25
2020Faster On-Device Training Using New Federated Momentum Algorithmcited by 25
2019Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learningcited by 18
2020Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobianscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 47; 2023: 57; 2024: 53; 2025: 27; 2026: 17.

210 papers.

Sort:

By area:

2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Modelscited by 203
2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 141
2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 121
2023Anti-DreamBooth: Protecting users from personalized text-to-image synthesiscited by 114
2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
2023A Brief Review of Hypernetworks in Deep Learningcited by 102
2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
2024Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resourcescited by 93

38 more

2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
2021Personalized Federated Learning with Gaussian Processescited by 76
2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
2021Personalized Federated Learning through Local Memorizationcited by 57
2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
2022FederatedScope: A Flexible Federated Learning Platform for Heterogeneitycited by 53
2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
2024Global and Local Prompts Cooperation via Optimal Transport for Federated Learningcited by 45
2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
2022Test-Time Robust Personalization for Federated Learningcited by 39
2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
2023Eliminating Domain Bias for Federated Learning in Representation Spacecited by 38
2023Personalized Federated Learning under Mixture of Distributionscited by 37
2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
2022FedTP: Federated Learning by Transformer Personalizationcited by 36
2023FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policycited by 36
2022DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalizationcited by 35
2022On the Convergence of Clustered Federated Learningcited by 31
2022Personalized Federated Learning With Graphcited by 29
2024FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuningcited by 27
2022FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classificationcited by 26
2024Harmonizing Generalization and Personalization in Federated Prompt Learningcited by 26
2023The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillationcited by 25
2024FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Datacited by 25
2022Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarkscited by 24
2024Privacy Preserving Prompt Engineering: A Surveycited by 24
2023Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedupcited by 23
2023Adaptive Test-Time Personalization for Federated Learningcited by 23
2023FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregationcited by 22
2023PeFLL: Personalized Federated Learning by Learning to Learncited by 21
2023Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shiftscited by 21
2023FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learningcited by 21
2024Federated Model Heterogeneous Matryoshka Representation Learningcited by 21
2023Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learningcited by 20
2023Dual Personalization on Federated Recommendationcited by 19
2023GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learningcited by 18
2022Federated Learning with Heterogeneous Architectures using Graph HyperNetworkscited by 17

50 most cited of 210 All 210 (JSON)