Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representations
Not on the map. Why?
Abstract (arXiv)
Bayesian personalized federated learning (BPFL) addresses challenges in existing personalized FL (PFL). BPFL aims to quantify the uncertainty and heterogeneity within and across clients towards uncertainty representations by addressing the statistical heterogeneity of client data. In PFL, some recent preliminary work proposes to decompose hidden neural representations into shared and local components and demonstrates interesting results. However, most of them do not address client uncertainty and heterogeneity in FL systems, while appropriately decoupling neural representations is challenging and often ad hoc. In this paper, we make the first attempt to introduce a general BPFL framework to decompose and jointly learn shared and personalized uncertainty representations on statistically heterogeneous client data over time. A Bayesian federated neural network BPFed instantiates BPFL by jointly learning cross-client shared uncertainty and client-specific personalized uncertainty over statistically heterogeneous and randomly participating clients. We further involve continual updating of prior distribution in BPFed to speed up the convergence and avoid catastrophic forgetting. Theoretical analysis and guarantees are provided in addition to the experimental evaluation of BPFed against the diversified baselines.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 27 Sep 2023.
Builds on
35 of 48 references.
References · 35
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2016Variational Inference: A Review for Statisticianscited by 1,466
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2018Federated Learning with Non-IID Datacited by 1,099
- 2017Federated Multi-Task Learningcited by 669
- 2020Personalized Federated Learning with Moreau Envelopescited by 612
- 2020Federated Learning with Matched Averagingcited by 564
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
- 2019Federated Learning with Personalization Layerscited by 533
23 more
- 2021Revisiting the Calibration of Modern Neural Networkscited by 387
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
- 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
- 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
- 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
- 2022Federated Learning with Partial Model Personalizationcited by 114
- 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
- 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
- 2021Personalized Federated Learning with Gaussian Processescited by 76
- 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
- 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
- 2018Posterior Concentration for Sparse Deep Learningcited by 50
- 2021FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learningcited by 43
- 2020Efficient Variational Inference for Sparse Deep Learning with Theoretical Guaranteecited by 37
- 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
- 2020Bayesian Few-Shot Classification with One-vs-Each P'olya-Gamma Augmented Gaussian Processescited by 32
- 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
- 2023Bayesian Federated Learning: A Surveycited by 22
- 2017On Statistical Optimality of Variational Bayescited by 21
- 2021On Data Efficiency of Meta-learningcited by 16
- 2021Lifelong Mixture of Variational Autoencoderscited by 10
Built on by
Citing papers per year: 2024: 2.
2 papers.