mlsanityData to 7 Oct 2026

Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theory

arXiv 2407.06935 ↗First posted 9 Jul 2024cs.LG stat.CO stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

This work introduces a novel and efficient Bayesian federated learning algorithm, namely, the Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC), for parameter estimation and uncertainty quantification. We establish rigorous convergence guarantees of FA-HMC on non-iid distributed data sets, under the strong convexity and Hessian smoothness assumptions. Our analysis investigates the effects of parameter space dimension, noise on gradients and momentum, and the frequency of communication (between the central node and local nodes) on the convergence and communication costs of FA-HMC. Beyond that, we establish the tightness of our analysis by showing that the convergence rate cannot be improved even for continuous FA-HMC process. Moreover, extensive empirical studies demonstrate that FA-HMC outperforms the existing Federated Averaging-Langevin Monte Carlo (FA-LD) algorithm.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 40 references that are in the map. First posted on arXiv 9 Jul 2024.

Builds on

23 of 40 references.

References · 23

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2019Deep Leakage from Gradientscited by 1,004
  6. 2020The Future of Digital Health with Federated Learningcited by 625
  7. 2020Federated Learning with Matched Averagingcited by 564
  8. 2018Deep Learning for Classical Japanese Literaturecited by 415
  9. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  10. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  11. 2015A Complete Recipe for Stochastic Gradient MCMCcited by 146
  12. 2017Underdamped Langevin MCMC: A non-asymptotic analysiscited by 131
11 more
  1. 2020Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Resultscited by 122
  2. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  3. 2019Stochastic gradient Markov chain Monte Carlocited by 56
  4. 2020On the Convergence of Langevin Monte Carlo: The Interplay between Tail Growth and Smoothnesscited by 55
  5. 2019Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradientscited by 51
  6. 2020Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Samplingcited by 32
  7. 2019Optimal Convergence Rate of Hamiltonian Monte Carlo for Strongly Logconcave Distributionscited by 28
  8. 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
  9. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  10. 2019Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learningcited by 8
  11. 2022Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithmscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Connecting Federated ADMM to Bayescited by 4
  2. 2025DIGing--SGLD: Decentralized and Scalable Langevin Sampling over Time--Varying Networkscited by 4
  3. 2025Morephy-Net: An Evolutionary Multi-objective Optimization for Replica-Exchange-based Physics-informed Neural Operator Learning Networkscited by 3
  4. 2025Measuring the right thing: justifying metrics in AI impact assessmentscited by 0