FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning
Not on the map. Why?
Abstract (arXiv)
Federated learning aims to collaboratively train a strong global model by accessing users' locally trained models but not their own data. A crucial step is therefore to aggregate local models into a global model, which has been shown challenging when users have non-i.i.d. data. In this paper, we propose a novel aggregation algorithm named FedBE, which takes a Bayesian inference perspective by sampling higher-quality global models and combining them via Bayesian model Ensemble, leading to much robust aggregation. We show that an effective model distribution can be constructed by simply fitting a Gaussian or Dirichlet distribution to the local models. Our empirical studies validate FedBE's superior performance, especially when users' data are not i.i.d. and when the neural networks go deeper. Moreover, FedBE is compatible with recent efforts in regularizing users' model training, making it an easily applicable module: you only need to replace the aggregation method but leave other parts of your federated learning algorithm intact. Our code is publicly available at https://github.com/hongyouc/FedBE.
Key facts. Cited by 176 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Sep 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 23; 2022: 33; 2023: 40; 2024: 41; 2025: 25; 2026: 14.
176 papers.
Sort:
By area:
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
- 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
- 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
- 2023Deep Model Fusion: A Surveycited by 89
- 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
- 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
- 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
- 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
- 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
38 more
- 2022Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Featurescited by 63
- 2021What Do We Mean by Generalization in Federated Learning?cited by 62
- 2023Revisiting Weighted Aggregation in Federated Learning with Neural Networkscited by 57
- 2023Better Generative Replay for Continual Federated Learningcited by 53
- 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
- 2023FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracycited by 49
- 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
- 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
- 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
- 2023Personalized Federated Learning under Mixture of Distributionscited by 37
- 2021Emerging Trends in Federated Learning: From Model Fusion to Federated X Learningcited by 35
- 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
- 2024An Aggregation-Free Federated Learning for Tackling Data Heterogeneitycited by 33
- 2022RSCFed: Random Sampling Consensus Federated Semi-supervised Learningcited by 32
- 2022Knowledge Distillation for Federated Learning: a Practical Guidecited by 31
- 2022Architecture Agnostic Federated Learning for Neural Networkscited by 30
- 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matterscited by 27
- 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
- 2023Teacher-Student Architecture for Knowledge Distillation: A Surveycited by 26
- 2021Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learningcited by 23
- 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
- 2023Adaptive Test-Time Personalization for Federated Learningcited by 23
- 2022DYNAFED: Tackling Client Data Heterogeneity with Global Dynamicscited by 22
- 2023Bayesian Federated Learning: A Surveycited by 22
- 2023FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregationcited by 22
- 2023Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learningcited by 20
- 2021Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentationcited by 19
- 2021Federated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Settingcited by 19
- 2022FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillationcited by 19
- 2025FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectorscited by 19
- 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
- 2022Self-Aware Personalized Federated Learningcited by 16
- 2022Federated Self-supervised Learning for Heterogeneous Clientscited by 16
- 2024FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Modelscited by 16
- 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
- 2021Neural Tangent Kernel Empowered Federated Learningcited by 15
- 2023PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guaranteescited by 15
- 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
50 most cited of 176 All 176 (JSON)
- 2026RoboFL: Federated Expert Assembly for World Action Models28 Sep 2026
- 2026Certified Uncertainty Propagation in One-Shot Federated Bayesian Models via Posterior Event Transport14 Sep 2026
- 2026Socialized Detector Learning: Trajectory-Guided and Reciprocal Distillation for Heterogeneous Object Detectors26 Aug 2026
- 2026FedQHD: Closed-Form Function-Space Federated Reinforcement Learning27 May 2026
- 2026Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning17 May 2026
- 2026Learned Neighbor Trust for Collaborative Deployment in Model-Agnostic Decentralized Learning6 May 2026
- 2026DeRelayL: Sustainable Decentralized Relay Learning30 Apr 2026
- 2026F3DGS: Federated 3D Gaussian Splatting for Decentralized Multi-Agent World Modeling2 Apr 2026
- 2026Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization18 Mar 2026
- 2026ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning19 Feb 2026
- 2026SCENE OTA-FD: Self-Centering Noncoherent Estimator for Over-the-Air Federated Distillation17 Feb 2026
- 2026FedEMA-Distill: Exponential Moving Average Guided Knowledge Distillation for Robust Federated Learning15 Feb 2026
38 more
- 2026Uncertainty Quantification in Federated Granger Causality Learning13 Feb 2026
- 2026Feature-Aware One-Shot Federated Learning via Hierarchical Token Sequences7 Jan 2026
- 2025On the Convergence and Stability of Distributed Sub-model Training8 Nov 2025
- 2025FedSM: Robust Semantics-Guided Feature Mixup for Bias Reduction in Federated Learning with Long-Tail Data31 Oct 2025
- 2025Federated Learning via Meta-Variational Dropout23 Oct 2025
- 2025High-Energy Concentration for Federated Learning in Frequency Domain16 Sep 2025
- 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning12 Sep 2025
- 2025Dual-Distilled Heterogeneous Federated Learning with Adaptive Margins for Trainable Global Prototypes26 Aug 2025
- 2025SHEFL: Sparse Heterogeneous Ensemble Federated Learning with Group-Balanced Aggregation12 Aug 2025
- 2025Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learning14 Jul 2025
- 2025FedSC: Federated Learning with Semantic-Aware Collaboration26 Jun 2025
- 2025FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Data25 Jun 2025
- 2025FedSKC: Federated Learning with Non-IID Data via Structural Knowledge Collaboration25 May 2025
- 2025ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoning22 May 2025
- 2025Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation20 May 2025
- 2025FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments19 May 2025
- 2025Bayesian Federated Cause-of-Death Classification and Quantification Under Distribution Shift4 May 2025
- 2025Guided Model Merging for Hybrid Data Learning: Leveraging Centralized Data to Refine Decentralized Models26 Mar 2025
- 2025Distributionally Robust Federated Learning: An ADMM Algorithm24 Mar 2025
- 2025FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors20 Mar 2025
- 2025FedBEns: One-Shot Federated Learning based on Bayesian Ensemble19 Mar 2025
- 2025FedLWS: Federated Learning with Adaptive Layer-wise Weight Shrinking19 Mar 2025
- 2025BTFL: A Bayesian-based Test-Time Generalization Method for Internal and External Data Distributions in Federated learning9 Mar 2025
- 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning9 Mar 2025
- 2025Multi-Attribute Multi-Grained Adaptation of Pre-Trained Language Models for Text Understanding from Bayesian Perspective8 Mar 2025
- 2025Federated Inverse Probability Treatment Weighting for Individual Treatment Effect Estimation6 Mar 2025
- 2025Robust Knowledge Distillation in Federated Learning: Counteracting Backdoor Attacks1 Feb 2025
- 2024FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis24 Dec 2024
- 2024Information-Geometric Barycenters for Bayesian Federated Learning16 Dec 2024
- 2024Effectiveness of Distributed Gradient Descent with Local Steps for Overparameterized Models10 Dec 2024
- 2024Tackling Data Heterogeneity in Federated Time Series Forecasting24 Nov 2024
- 2024Federated Learning in Chemical Engineering: A Tutorial on a Framework for Privacy-Preserving Collaboration Across Distributed Data Sources23 Nov 2024
- 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions19 Nov 2024
- 2024Federated Contrastive Learning of Graph-Level Representations18 Nov 2024
- 2024Revisiting Ensembling in One-Shot Federated Learning11 Nov 2024
- 2024A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning28 Oct 2024
- 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion27 Oct 2024
- 2024Personalizing Low-Rank Bayesian Neural Networks Via Federated Learning18 Oct 2024
50 newest of 176 All 176 (JSON)
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
- 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
- 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
- 2023Deep Model Fusion: A Surveycited by 89
- 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
- 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
- 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
- 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
- 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
12 most cited of 171 All 171 (JSON)
- 2026Certified Uncertainty Propagation in One-Shot Federated Bayesian Models via Posterior Event Transport14 Sep 2026
- 2026Socialized Detector Learning: Trajectory-Guided and Reciprocal Distillation for Heterogeneous Object Detectors26 Aug 2026
- 2026FedQHD: Closed-Form Function-Space Federated Reinforcement Learning27 May 2026
- 2026Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning17 May 2026
- 2026Learned Neighbor Trust for Collaborative Deployment in Model-Agnostic Decentralized Learning6 May 2026
- 2026DeRelayL: Sustainable Decentralized Relay Learning30 Apr 2026
- 2026Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization18 Mar 2026
- 2026ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning19 Feb 2026
- 2026SCENE OTA-FD: Self-Centering Noncoherent Estimator for Over-the-Air Federated Distillation17 Feb 2026
- 2026FedEMA-Distill: Exponential Moving Average Guided Knowledge Distillation for Robust Federated Learning15 Feb 2026
- 2026Uncertainty Quantification in Federated Granger Causality Learning13 Feb 2026
- 2026Feature-Aware One-Shot Federated Learning via Hierarchical Token Sequences7 Jan 2026
12 newest of 171 All 171 (JSON)