mlsanityData to 14 Oct 2026

FedHB: Hierarchical Bayesian Federated Learning

arXiv 2305.04979 ↗First posted 8 May 2023cs.LG cs.DC stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

We propose a novel hierarchical Bayesian approach to Federated Learning (FL), where our model reasonably describes the generative process of clients' local data via hierarchical Bayesian modeling: constituting random variables of local models for clients that are governed by a higher-level global variate. Interestingly, the variational inference in our Bayesian model leads to an optimisation problem whose block-coordinate descent solution becomes a distributed algorithm that is separable over clients and allows them not to reveal their own private data at all, thus fully compatible with FL. We also highlight that our block-coordinate algorithm has particular forms that subsume the well-known FL algorithms including Fed-Avg and Fed-Prox as special cases. Beyond introducing novel modeling and derivations, we also offer convergence analysis showing that our block-coordinate FL algorithm converges to an (local) optimum of the objective at the rate of O(1/√t), the same rate as regular (centralised) SGD, as well as the generalisation error analysis where we prove that the test error of our model on unseen data is guaranteed to vanish as we increase the training data size, thus asymptotically optimal.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 39 of its 73 references that are in the map. First posted on arXiv 8 May 2023.

Builds on

39 of 73 references.

References · 39

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  6. 2016Variational Inference: A Review for Statisticianscited by 1,472
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  9. 2017Federated Multi-Task Learningcited by 672
  10. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 562
27 more
  1. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  2. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  3. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
  4. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  5. 2015Coordinate Descent Algorithmscited by 219
  6. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 192
  7. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  8. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 182
  9. 2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
  10. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 155
  11. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  12. 2021Personalized Federated Learning with Gaussian Processescited by 76
  13. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  14. 2016A Strongly Quasiconvex PAC-Bayesian Boundcited by 54
  15. 2021Federated Learning with Domain Generalizationcited by 41
  16. 2020Efficient Variational Inference for Sparse Deep Learning with Theoretical Guaranteecited by 37
  17. 2019PAC-Bayes with Backpropcited by 36
  18. 2019Private Federated Learning with Domain Adaptationcited by 36
  19. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  20. 2019Convergence Rates of Variational Inference in Sparse Deep Learningcited by 27
  21. 2021GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learningcited by 26
  22. 2017On Statistical Optimality of Variational Bayescited by 21
  23. 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
  24. 2022Partitioned Variational Inference: A Framework for Probabilistic Federated Learningcited by 13
  25. 2021DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbscited by 11
  26. 2021An Expectation-Maximization Perspective on Federated Learningcited by 9
  27. 2014Randomized Block Coordinate Descent for Online and Stochastic Optimizationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 2.

8 papers.

  1. 2023Dirichlet-based Uncertainty Quantification for Personalized Federated Learning with Improved Posterior Networkscited by 8
  2. 2023Federated Learning with Nonvacuous Generalisation Boundscited by 2
  3. 2024Information-Geometric Barycenters for Bayesian Federated Learningcited by 1
  4. 2025Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregationcited by 1
  5. 2024Your Data, My Model: Learning Who Really Helps in Federated Learningcited by 0
  6. 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learningcited by 0
  7. 2026A Hierarchical Sampling Framework for bounding the Generalization Error of Federated Learningcited by 0
  8. 2026Range Penalization: Theoretical Insights with Applications in Federated Learningcited by 0