# Federated ADMM from Bayesian Duality

arXiv: https://arxiv.org/abs/2506.13150 · HTML: https://mlsanity.com/paper/2506.13150/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 75 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data](https://mlsanity.com/paper/2005.11418/) (2020)
- [Generalized Federated Learning via Sharpness Aware Minimization](https://mlsanity.com/paper/2206.02618/) (2022)
- [ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!](https://mlsanity.com/paper/2202.09357/) (2022)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms](https://mlsanity.com/paper/2010.05273/) (2020)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Variational Learning is Effective for Large Deep Networks](https://mlsanity.com/paper/2402.17641/) (2024)
- [The Information Geometry of Mirror Descent](https://mlsanity.com/paper/1310.7780/) (2013)
- [Federated Learning via Inexact ADMM](https://mlsanity.com/paper/2204.10607/) (2022)
- [SAM as an Optimal Relaxation of Bayes](https://mlsanity.com/paper/2210.01620/) (2022)
- [Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models](https://mlsanity.com/paper/1807.04489/) (2018)
- [FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participation](https://mlsanity.com/paper/2203.15104/) (2022)
- [FedPop: A Bayesian Approach for Personalised Federated Learning](https://mlsanity.com/paper/2206.03611/) (2022)
- [Practical Large-Scale Linear Programming using Primal-Dual Hybrid Gradient](https://mlsanity.com/paper/2106.04756/) (2021)
- [Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization](https://mlsanity.com/paper/2405.18890/) (2024)
- [FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity](https://mlsanity.com/paper/2204.03529/) (2022)
- [Bregman Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1306.3203/) (2013)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach](https://mlsanity.com/paper/2302.04228/) (2023)
- [Partitioned Variational Inference: A Framework for Probabilistic Federated Learning](https://mlsanity.com/paper/2202.12275/) (2022)
- [Adaptive ADMM with Spectral Penalty Parameter Selection](https://mlsanity.com/paper/1605.07246/) (2016)
- [An Expectation-Maximization Perspective on Federated Learning](https://mlsanity.com/paper/2111.10192/) (2021)

## Built on by (most cited first, 3 of 3)

- [Knowledge Adaptation as Posterior Correction](https://mlsanity.com/paper/2506.14262/) (2025)
- [SOAP-Bubbles: Structured Weight Uncertainty for Neural Networks](https://mlsanity.com/paper/2606.23357/) (2026)
- [A Generalization of Amari's Bayesian Duality](https://mlsanity.com/paper/2609.09126/) (2026)

## Abstract (arXiv)

We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objectives are associated with a duality structure that not only resembles the structure of ADMM's fixed-points but also generalizes it. For example, ADMM-like updates are recovered when the VB objective is optimized over the isotropic-Gaussian family, and new non-trivial extensions are obtained for other exponential-family distributions. These extensions include a Newton-like variant that converges in one step on quadratic objectives and an Adam-like variant that yields up to 7% accuracy boosts for deep heterogeneous cases. Our work opens a new Bayesian way to generalize ADMM and other primal-dual methods.