A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy
Not on the map. Why?
Abstract (arXiv)
A distinguishing characteristic of federated learning is that the (local) client data could have statistical heterogeneity. This heterogeneity has motivated the design of personalized learning, where individual (personalized) models are trained, through collaboration. There have been various personalization methods proposed in literature, with seemingly very different forms and methods ranging from use of a single global model for local regularization and model interpolation, to use of multiple global models for personalized clustering, etc. In this work, we begin with a generative framework that could potentially unify several different algorithms as well as suggest new algorithms. We apply our generative framework to personalized estimation, and connect it to the classical empirical Bayes' methodology. We develop private personalized estimation under this framework. We then use our generative framework for learning, which unifies several known personalized FL algorithms and also suggests new ones; we propose and study a new algorithm AdaPeD based on a Knowledge Distillation, which numerically outperforms several known algorithms. We also develop privacy for personalized learning methods with guarantees for user-level privacy and composition. We numerically evaluate the performance as well as the privacy for both the estimation and learning problems, demonstrating the advantages of our proposed methods.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 62 references that are in the map. First posted on arXiv 5 Jul 2022.
Builds on
39 of 62 references.
References · 39
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018LEAF: A Benchmark for Federated Settingscited by 780
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
- 2017Federated Multi-Task Learningcited by 669
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
- 2017Renyi Differential Privacycited by 643
- 2020Personalized Federated Learning with Moreau Envelopescited by 612
- 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
- 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
27 more
- 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
- 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
- 2020Adaptive Personalized Federated Learningcited by 353
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
- 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
- 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
- 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
- 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
- 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
- 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
- 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
- 2020The Discrete Gaussian for Differential Privacycited by 134
- 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
- 2019Hypothesis Testing Interpretations and Renyi Differential Privacycited by 84
- 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
- 2021Learning with User-Level Privacycited by 72
- 2020Federated Mutual Learningcited by 70
- 2016Better Guarantees for k-Means and Euclidean k-Median by Primal-Dual Algorithmscited by 54
- 2020Learning discrete distributions: user vs item-level privacycited by 41
- 2019Differentially Private Meta-Learningcited by 40
- 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 34
- 2020Differentially Private Clustering: Tight Approximation Ratioscited by 34
- 2021QuPeD: Quantized Personalization via Distillation with Applications to Federated Learningcited by 32
- 2021On the Renyi Differential Privacy of the Shuffle Modelcited by 16
- 2019Locally Private k-Means Clusteringcited by 15
- 2019Maximum Likelihood Estimation for Learning Populations of Parameterscited by 11
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.