# PeFLL: Personalized Federated Learning by Learning to Learn

arXiv: https://arxiv.org/abs/2306.05515 · HTML: https://mlsanity.com/paper/2306.05515/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 63 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 30 of 37)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Federated Learning with Personalization Layers](https://mlsanity.com/paper/1912.00818/) (2019)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [Improving Federated Learning Personalization via Model Agnostic Meta Learning](https://mlsanity.com/paper/1909.12488/) (2019)
- [Personalized Federated Learning using Hypernetworks](https://mlsanity.com/paper/2103.04628/) (2021)
- [Federated Learning of a Mixture of Global and Local Models](https://mlsanity.com/paper/2002.05516/) (2020)
- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [Lower Bounds and Optimal Algorithms for Personalized Federated Learning](https://mlsanity.com/paper/2010.02372/) (2020)
- [Tighter risk certificates for neural networks](https://mlsanity.com/paper/2007.12911/) (2020)
- [Layer-wised Model Aggregation for Personalized Federated Learning](https://mlsanity.com/paper/2205.03993/) (2022)
- [PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees](https://mlsanity.com/paper/2002.05551/) (2020)
- [Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory](https://mlsanity.com/paper/1711.01244/) (2017)

## Built on by (most cited first, 21 of 21)

- [Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations](https://mlsanity.com/paper/2405.09839/) (2024)
- [More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms](https://mlsanity.com/paper/2402.04054/) (2024)
- [From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learning](https://mlsanity.com/paper/2501.19067/) (2025)
- [FedCoSR: Personalized Federated Learning with Contrastive Shareable Representations for Label Heterogeneity in Non-IID Data](https://mlsanity.com/paper/2404.17916/) (2024)
- [Improved Modelling of Federated Datasets using Mixtures-of-Dirichlet-Multinomials](https://mlsanity.com/paper/2406.02416/) (2024)
- [Bad-PFL: Exploring Backdoor Attacks against Personalized Federated Learning](https://mlsanity.com/paper/2501.12736/) (2025)
- [Fast Rate Bounds for Multi-Task and Meta-Learning with Different Sample Sizes](https://mlsanity.com/paper/2505.15496/) (2025)
- [Deploying Models to Non-participating Clients in Federated Learning without Fine-tuning: A Hypernetwork-based Approach](https://mlsanity.com/paper/2508.12673/) (2025)
- [Optimized Local Updates in Federated Learning via Reinforcement Learning](https://mlsanity.com/paper/2506.06337/) (2025)
- [Hypernetworks for Model-Heterogeneous Personalized Federated Learning](https://mlsanity.com/paper/2507.22330/) (2025)
- [Personalized Subgraph Federated Learning with Sheaf Collaboration](https://mlsanity.com/paper/2508.13642/) (2025)
- [Lazy But Effective: Collaborative Personalized Federated Learning with Heterogeneous Data](https://mlsanity.com/paper/2505.02540/) (2025)
- [Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions](https://mlsanity.com/paper/2505.15579/) (2025)
- [Onboarding Without Forgetting: Hypernetwork Personalization with Data-Free Replay for Personalized Federated Learning](https://mlsanity.com/paper/2508.05157/) (2025)
- [Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation](https://mlsanity.com/paper/2509.25906/) (2025)
- [AutoFed: Personalized Federated Traffic Prediction via Adaptive Prompt](https://mlsanity.com/paper/2512.24625/) (2025)
- [Heterogeneity-Aware Client Selection Methodology For Efficient Federated Learning](https://mlsanity.com/paper/2602.20450/) (2026)
- [Representation-Aligned Multi-Scale Personalization for Federated Learning](https://mlsanity.com/paper/2604.11278/) (2026)
- [FedCoE: Bridging Generalization and Personalization via Federated Coordinated Dual-level MoEs](https://mlsanity.com/paper/2605.21264/) (2026)
- [Similarity-Aware Personalized Federated Learning in Heterogeneous Environments](https://mlsanity.com/paper/2609.02241/) (2026)
- [Privacy-Aligned Personalized Federated Learning with Compact Adaptation and Variable-Length Gaussian Communication](https://mlsanity.com/paper/2609.15950/) (2026)

## Abstract (arXiv)

We present PeFLL, a new personalized federated learning algorithm that improves over the state-of-the-art in three aspects: 1) it produces more accurate models, especially in the low-data regime, and not only for clients present during its training phase, but also for any that may emerge in the future; 2) it reduces the amount of on-client computation and client-server communication by providing future clients with ready-to-use personalized models that require no additional finetuning or optimization; 3) it comes with theoretical guarantees that establish generalization from the observed clients to future ones. At the core of PeFLL lies a learning-to-learn approach that jointly trains an embedding network and a hypernetwork. The embedding network is used to represent clients in a latent descriptor space in a way that reflects their similarity to each other. The hypernetwork takes as input such descriptors and outputs the parameters of fully personalized client models. In combination, both networks constitute a learning algorithm that achieves state-of-the-art performance in several personalized federated learning benchmarks.