Federated Variational Inference: Towards Improved Personalization and Generalization
Not on the map. Why?
Abstract (arXiv)
Conventional federated learning algorithms train a single global model by leveraging all participating clients' data. However, due to heterogeneity in client generative distributions and predictive models, these approaches may not appropriately approximate the predictive process, converge to an optimal state, or generalize to new clients. We study personalization and generalization in stateless cross-device federated learning setups assuming heterogeneity in client data distributions and predictive models. We first propose a hierarchical generative model and formalize it using Bayesian Inference. We then approximate this process using Variational Inference to train our model efficiently. We call this algorithm Federated Variational Inference (FedVI). We use PAC-Bayes analysis to provide generalization bounds for FedVI. We evaluate our model on FEMNIST and CIFAR-100 image classification and show that FedVI beats the state-of-the-art on both tasks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 23 May 2023.
Builds on
30 of 51 references.
References · 30
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2016Variational Inference: A Review for Statisticianscited by 1,466
- 2020Adaptive Federated Optimizationcited by 1,036
- 2016Deep Variational Information Bottleneckcited by 915
- 2018LEAF: A Benchmark for Federated Settingscited by 780
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
18 more
- 2020Adaptive Personalized Federated Learningcited by 353
- 2021A Field Guide to Federated Optimizationcited by 312
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
- 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
- 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
- 2016PAC-Bayesian Theory Meets Bayesian Inferencecited by 87
- 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
- 2021Federated Reconstruction: Partially Local Federated Learningcited by 78
- 2019Variational Federated Multi-Task Learningcited by 69
- 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
- 2021What Do We Mean by Generalization in Federated Learning?cited by 62
- 2021Personalized Federated Learning through Local Memorizationcited by 57
- 2019Meta Matrix Factorization for Federated Rating Predictionscited by 48
- 2016Differential Privacy as a Mutual Information Constraintcited by 41
- 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
- 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
- 2019Variational Predictive Information Bottleneckcited by 8
- 2023Federated Variational Inference Methods for Structured Latent Variable Modelscited by 4
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.