Personalized Federated Learning via Variational Bayesian Inference
Not on the map. Why?
Abstract (arXiv)
Federated learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, this paper proposes a novel personalized federated learning method via Bayesian variational inference named pFedBayes. To alleviate the overfitting, weight uncertainty is introduced to neural networks for clients and the server. To achieve personalization, each client updates its local distribution parameters by balancing its construction error over private data and its KL divergence with global distribution from the server. Theoretical analysis gives an upper bound of averaged generalization error and illustrates that the convergence rate of the generalization error is minimax optimal up to a logarithmic factor. Experiments show that the proposed method outperforms other advanced personalized methods on personalized models, e.g., pFedBayes respectively outperforms other SOTA algorithms by 1.25%, 0.42% and 11.71% on MNIST, FMNIST and CIFAR-10 under non-i.i.d. limited data.
Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 37 of its 56 references that are in the map. First posted on arXiv 16 Jun 2022.
Builds on
37 of 56 references.
References · 37
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2016Variational Inference: A Review for Statisticianscited by 1,466
- 2017Federated Multi-Task Learningcited by 669
- 2020Personalized Federated Learning with Moreau Envelopescited by 612
- 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
- 2015Weight Uncertainty in Neural Networkscited by 554
- 2019Federated Learning with Personalization Layerscited by 533
25 more
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
- 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
- 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
- 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 336
- 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
- 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
- 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
- 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
- 2019Practical Deep Learning with Bayesian Principlescited by 158
- 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
- 2019One-Shot Federated Learningcited by 124
- 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
- 2020FedDANE: A Federated Newton-Type Methodcited by 76
- 2021Personalized Federated Learning with Gaussian Processescited by 76
- 2017Concentration of tempered posteriors and of their variational approximationscited by 66
- 2018Posterior Concentration for Sparse Deep Learningcited by 50
- 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
- 2020Efficient Variational Inference for Sparse Deep Learning with Theoretical Guaranteecited by 37
- 2019Convergence Rates of Variational Inference in Sparse Deep Learningcited by 27
- 2022Federated Learning with Position-Aware Neuronscited by 24
- 2017On Statistical Optimality of Variational Bayescited by 21
- 2019Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learningcited by 18
- 2018Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixturescited by 17
- 2022Sparse Federated Learning with Hierarchical Personalized Modelscited by 4
Built on by
Citing papers per year: 2022: 5; 2023: 21; 2024: 25; 2025: 12; 2026: 5.
68 papers.
Sort:
By area:
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
- 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
- 2023Bayesian Federated Learning: A Surveycited by 22
- 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
- 2023FedBone: Towards Large-Scale Federated Multi-Task Learningcited by 14
- 2024Modeling Inter-Intra Heterogeneity for Graph Federated Learningcited by 12
- 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
- 2024Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimationcited by 9
- 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
- 2023PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learningcited by 8
- 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
38 more
- 2024Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learningcited by 7
- 2024MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classescited by 7
- 2022How to Combine Variational Bayesian Networks in Federated Learningcited by 5
- 2023Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clusteringcited by 5
- 2024Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approachcited by 5
- 2024Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagationcited by 5
- 2022Avoid Overfitting User Specific Information in Federated Keyword Spottingcited by 4
- 2022Differentiated Federated Reinforcement Learning Based Traffic Offloading on Space-Air-Ground Integrated Networkscited by 4
- 2023Federated Variational Inference Methods for Structured Latent Variable Modelscited by 4
- 2023FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropoutcited by 4
- 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
- 2024Federated Bayesian Deep Learning: The Application of Statistical Aggregation Methods to Bayesian Modelscited by 3
- 2024Federated Neural Nonparametric Point Processescited by 3
- 2024A Bayesian Framework for Clustered Federated Learningcited by 3
- 2025Personalized Language Model Learning on Text Data Without User Identifierscited by 3
- 2023Federated Variational Inference: Towards Improved Personalization and Generalizationcited by 2
- 2023Privacy Preserving Bayesian Federated Learning in Heterogeneous Settingscited by 2
- 2023Federated Neuro-Symbolic Learningcited by 2
- 2023Federated PAC-Bayesian Learning on Non-IID datacited by 2
- 2023Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representationscited by 2
- 2023Federated Learning with Nonvacuous Generalisation Boundscited by 2
- 2023Federated learning-outcome prediction with multi-layer privacy protectioncited by 2
- 2024Future-Proofing Medical Imaging with Privacy-Preserving Federated Learning and Uncertainty Quantification: A Reviewcited by 2
- 2024Asymmetrical Reciprocity-based Federated Learning for Resolving Disparities in Medical Diagnosiscited by 2
- 2022Tensor Decomposition based Personalized Federated Learningcited by 1
- 2022Personalized Federated Learning with Hidden Information on Personalized Priorcited by 1
- 2023Personalized Federated Learning via Amortized Bayesian Meta-Learningcited by 1
- 2023Distributed Variational Inference for Online Supervised Learningcited by 1
- 2023PPFL: A Personalized Federated Learning Framework for Heterogeneous Populationcited by 1
- 2024Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tunecited by 1
- 2024Bayesian Federated Model Compression for Communication and Computation Efficiencycited by 1
- 2024FedMAP: Personalised Federated Learning for Real Large-Scale Healthcare Systemscited by 1
- 2024Semi-Variance Reduction for Fair Federated Learningcited by 1
- 2024Federated Learning with Uncertainty and Personalization via Efficient Second-order Optimizationcited by 1
- 2024Task Diversity in Bayesian Federated Learning: Simultaneous Processing of Classification and Regressioncited by 1
- 2024Information-Geometric Barycenters for Bayesian Federated Learningcited by 1
- 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Frameworkcited by 1
- 2025Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregationcited by 1
50 most cited of 68 All 68 (JSON)
- 2026Certified Uncertainty Propagation in One-Shot Federated Bayesian Models via Posterior Event Transport14 Sep 2026
- 2026Communication-Efficient Personalized Federated Learning via Layer-Wise Multi-Threshold Random Sketching4 Sep 2026
- 2026Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization18 Mar 2026
- 2026Uncertainty Quantification in Federated Granger Causality Learning13 Feb 2026
- 2026BlindU: Blind Machine Unlearning without Revealing Erasing Data12 Jan 2026
- 2025Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching17 Nov 2025
- 2025Bayesian Coreset Optimization for Personalized Federated Learning3 Nov 2025
- 2025Federated Learning via Meta-Variational Dropout23 Oct 2025
- 2025CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks23 Oct 2025
- 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning12 Sep 2025
- 2025Towards Instance-wise Personalized Federated Learning via Semi-Implicit Bayesian Prompt Tuning27 Aug 2025
- 2025FIVA: Federated Inverse Variance Averaging for Universal CT Segmentation with Uncertainty Estimation8 Aug 2025
38 more
- 2025Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation20 May 2025
- 2025Bayesian Federated Cause-of-Death Classification and Quantification Under Distribution Shift4 May 2025
- 2025Guided Model Merging for Hybrid Data Learning: Leveraging Centralized Data to Refine Decentralized Models26 Mar 2025
- 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework2 Feb 2025
- 2025Personalized Language Model Learning on Text Data Without User Identifiers10 Jan 2025
- 2024Asymmetrical Reciprocity-based Federated Learning for Resolving Disparities in Medical Diagnosis27 Dec 2024
- 2024Information-Geometric Barycenters for Bayesian Federated Learning16 Dec 2024
- 2024Modeling Inter-Intra Heterogeneity for Graph Federated Learning16 Dec 2024
- 2024Task Diversity in Bayesian Federated Learning: Simultaneous Processing of Classification and Regression14 Dec 2024
- 2024Federated Learning with Uncertainty and Personalization via Efficient Second-order Optimization27 Nov 2024
- 2024DUA-D2C: Dynamic Uncertainty Aware Method for Overfitting Remediation in Deep Learning24 Nov 2024
- 2024A Bayesian Framework for Clustered Federated Learning20 Oct 2024
- 2024Personalizing Low-Rank Bayesian Neural Networks Via Federated Learning18 Oct 2024
- 2024Federated Neural Nonparametric Point Processes8 Oct 2024
- 2024Future-Proofing Medical Imaging with Privacy-Preserving Federated Learning and Uncertainty Quantification: A Review24 Sep 2024
- 2024Personalized Multi-tier Federated Learning19 Jul 2024
- 2024Semi-Variance Reduction for Fair Federated Learning23 Jun 2024
- 2024Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimation18 Jun 2024
- 2024FedMAP: Personalised Federated Learning for Real Large-Scale Healthcare Systems29 May 2024
- 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantification24 Apr 2024
- 2024MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes14 Apr 2024
- 2024Bayesian Federated Model Compression for Communication and Computation Efficiency11 Apr 2024
- 2024Federated Bayesian Deep Learning: The Application of Statistical Aggregation Methods to Bayesian Models22 Mar 2024
- 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications3 Mar 2024
- 2024Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune25 Feb 2024
- 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI1 Feb 2024
- 2024Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagation29 Jan 2024
- 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data17 Jan 2024
- 2024Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach16 Jan 2024
- 2024Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learning16 Jan 2024
- 2023Federated learning-outcome prediction with multi-layer privacy protection25 Dec 2023
- 2023PPFL: A Personalized Federated Learning Framework for Heterogeneous Population22 Oct 2023
- 2023Federated Learning with Nonvacuous Generalisation Bounds17 Oct 2023
- 2023PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning13 Oct 2023
- 2023Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representations27 Sep 2023
- 2023Federated PAC-Bayesian Learning on Non-IID data13 Sep 2023
- 2023Distributed Variational Inference for Online Supervised Learning5 Sep 2023
- 2023Federated Neuro-Symbolic Learning29 Aug 2023
50 newest of 68 All 68 (JSON)
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
- 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
- 2023Bayesian Federated Learning: A Surveycited by 22
- 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
- 2023FedBone: Towards Large-Scale Federated Multi-Task Learningcited by 14
- 2024Modeling Inter-Intra Heterogeneity for Graph Federated Learningcited by 12
- 2024FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Datacited by 10
- 2024Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimationcited by 9
- 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
- 2023PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learningcited by 8
- 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
12 most cited of 67 All 67 (JSON)
- 2026Certified Uncertainty Propagation in One-Shot Federated Bayesian Models via Posterior Event Transport14 Sep 2026
- 2026Communication-Efficient Personalized Federated Learning via Layer-Wise Multi-Threshold Random Sketching4 Sep 2026
- 2026Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization18 Mar 2026
- 2026Uncertainty Quantification in Federated Granger Causality Learning13 Feb 2026
- 2025Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching17 Nov 2025
- 2025Bayesian Coreset Optimization for Personalized Federated Learning3 Nov 2025
- 2025Federated Learning via Meta-Variational Dropout23 Oct 2025
- 2025CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks23 Oct 2025
- 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning12 Sep 2025
- 2025Towards Instance-wise Personalized Federated Learning via Semi-Implicit Bayesian Prompt Tuning27 Aug 2025
- 2025FIVA: Federated Inverse Variance Averaging for Universal CT Segmentation with Uncertainty Estimation8 Aug 2025
- 2025Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation20 May 2025
12 newest of 67 All 67 (JSON)