mlsanityData to 7 Oct 2026

Connecting Federated ADMM to Bayes

arXiv 2501.17325 ↗First posted 28 Jan 2025cs.LG cs.AI stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

We provide new connections between two distinct federated learning approaches based on (i) ADMM and (ii) Variational Bayes (VB), and propose new variants by combining their complementary strengths. Specifically, we show that the dual variables in ADMM naturally emerge through the 'site' parameters used in VB with isotropic Gaussian covariances. Using this, we derive two versions of ADMM from VB that use flexible covariances and functional regularisation, respectively. Through numerical experiments, we validate the improvements obtained in performance. The work shows connection between two fields that are believed to be fundamentally different and combines them to improve federated learning.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

33 of 47 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  6. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  7. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  9. 2014New insights and perspectives on the natural gradient methodcited by 435
  10. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  11. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  12. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
21 more
  1. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  2. 2019Functional Regularisation for Continual Learning with Gaussian Processescited by 121
  3. 2022FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settingscited by 121
  4. 2020Continual Deep Learning by Functional Regularisation of Memorable Pastcited by 104
  5. 2021The Bayesian Learning Rulecited by 95
  6. 2019Approximate Inference Turns Deep Networks into Gaussian Processescited by 89
  7. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  8. 2017Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Modelscited by 65
  9. 2024Variational Learning is Effective for Large Deep Networkscited by 62
  10. 2022Federated Learning via Inexact ADMMcited by 44
  11. 2018Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Modelscited by 37
  12. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 35
  13. 2020Handling the Positive-Definite Constraint in the Bayesian Learning Rulecited by 31
  14. 2018Partitioned Variational Inference: A unified framework encompassing federated and continual learningcited by 29
  15. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  16. 2021Knowledge-Adaptation Priorscited by 20
  17. 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
  18. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  19. 2022Partitioned Variational Inference: A Framework for Probabilistic Federated Learningcited by 13
  20. 2024Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theorycited by 4
  21. 2022Differentially private partitioned variational inferencecited by 2

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Knowledge Adaptation as Posterior Correctioncited by 6
  2. 2025Federated ADMM from Bayesian Dualitycited by 3
  3. 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Frameworkcited by 1
  4. 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learningcited by 0