mlsanityData to 14 Oct 2026

Federated ADMM from Bayesian Duality

arXiv 2506.13150 ↗First posted 16 Jun 2025cs.LG math.OC stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objectives are associated with a duality structure that not only resembles the structure of ADMM's fixed-points but also generalizes it. For example, ADMM-like updates are recovered when the VB objective is optimized over the isotropic-Gaussian family, and new non-trivial extensions are obtained for other exponential-family distributions. These extensions include a Newton-like variant that converges in one step on quadratic objectives and an Adam-like variant that yields up to 7% accuracy boosts for deep heterogeneous cases. Our work opens a new Bayesian way to generalize ADMM and other primal-dual methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 37 of its 75 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

37 of 75 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  4. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  5. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  6. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  7. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
  8. 2013Expectation Propagation for approximate Bayesian inferencecited by 174
  9. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 144
  10. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  11. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  12. 2021The Bayesian Learning Rulecited by 99
25 more
  1. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  2. 2017Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Modelscited by 65
  3. 2024Variational Learning is Effective for Large Deep Networkscited by 65
  4. 2013The Information Geometry of Mirror Descentcited by 54
  5. 2022Federated Learning via Inexact ADMMcited by 44
  6. 2022SAM as an Optimal Relaxation of Bayescited by 41
  7. 2018Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Modelscited by 38
  8. 2022FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participationcited by 36
  9. 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
  10. 2021Practical Large-Scale Linear Programming using Primal-Dual Hybrid Gradientcited by 32
  11. 2024Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimizationcited by 28
  12. 2022FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneitycited by 25
  13. 2013Bregman Alternating Direction Method of Multiplierscited by 19
  14. 2021Dual Parameterization of Sparse Variational Gaussian Processescited by 19
  15. 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
  16. 2022Partitioned Variational Inference: A Framework for Probabilistic Federated Learningcited by 13
  17. 2016Adaptive ADMM with Spectral Penalty Parameter Selectioncited by 10
  18. 2021An Expectation-Maximization Perspective on Federated Learningcited by 9
  19. 2014Convergence of Bregman alternating direction method with multipliers for nonconvex composite problemscited by 7
  20. 2015Parameter Selection and Pre-Conditioning for a Graph Form Solvercited by 4
  21. 2025Connecting Federated ADMM to Bayescited by 4
  22. 2023Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Methodcited by 3
  23. 2014An inertial alternating direction method of multiplierscited by 2
  24. 2015D-MFVI: Distributed Mean Field Variational Inference using Bregman ADMMcited by 1
  25. 2020Optimization of Graph Total Variation via Active-Set-based Combinatorial Reconditioningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Knowledge Adaptation as Posterior Correctioncited by 6
  2. 2026SOAP-Bubbles: Structured Weight Uncertainty for Neural Networkscited by 3
  3. 2026A Generalization of Amari's Bayesian Dualitycited by 0