# Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms

arXiv: https://arxiv.org/abs/2010.05273 · HTML: https://mlsanity.com/paper/2010.05273/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2020.

## Built on by (most cited first, 30 of 78)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models](https://mlsanity.com/paper/2305.18292/) (2023)
- [The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation](https://mlsanity.com/paper/2102.06387/) (2021)
- [Federated Learning with Label Distribution Skew via Logits Calibration](https://mlsanity.com/paper/2209.00189/) (2022)
- [Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark](https://mlsanity.com/paper/2311.06750/) (2023)
- [The Skellam Mechanism for Differentially Private Federated Learning](https://mlsanity.com/paper/2110.04995/) (2021)
- [Personalized Federated Learning via Variational Bayesian Inference](https://mlsanity.com/paper/2206.07977/) (2022)
- [What Do We Mean by Generalization in Federated Learning?](https://mlsanity.com/paper/2110.14216/) (2021)
- [Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing](https://mlsanity.com/paper/2106.04502/) (2021)
- [FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Data](https://mlsanity.com/paper/2211.09299/) (2022)
- [Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning](https://mlsanity.com/paper/2210.00226/) (2022)
- [A Bayesian Federated Learning Framework with Online Laplace Approximation](https://mlsanity.com/paper/2102.01936/) (2021)
- [Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective](https://mlsanity.com/paper/2111.03741/) (2021)
- [Multi-Level Branched Regularization for Federated Learning](https://mlsanity.com/paper/2207.06936/) (2022)
- [Relaxed Contrastive Learning for Federated Learning](https://mlsanity.com/paper/2401.04928/) (2024)
- [Understanding Generalization of Federated Learning via Stability: Heterogeneity Matters](https://mlsanity.com/paper/2306.03824/) (2023)
- [Communication-Efficient Federated Learning with Accelerated Client Gradient](https://mlsanity.com/paper/2201.03172/) (2022)
- [Bayesian Federated Learning: A Survey](https://mlsanity.com/paper/2304.13267/) (2023)
- [FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation](https://mlsanity.com/paper/2310.00339/) (2023)
- [SphereFed: Hyperspherical Federated Learning](https://mlsanity.com/paper/2207.09413/) (2022)
- [Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach](https://mlsanity.com/paper/2302.04228/) (2023)
- [Confidence-aware Personalized Federated Learning via Variational Expectation Maximization](https://mlsanity.com/paper/2305.12557/) (2023)
- [Self-Aware Personalized Federated Learning](https://mlsanity.com/paper/2204.08069/) (2022)
- [On Convergence of Federated Averaging Langevin Dynamics](https://mlsanity.com/paper/2112.05120/) (2021)
- [Federated Learning with Uncertainty via Distilled Predictive Distributions](https://mlsanity.com/paper/2206.07562/) (2022)
- [Modeling Inter-Intra Heterogeneity for Graph Federated Learning](https://mlsanity.com/paper/2412.11402/) (2024)
- [Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning](https://mlsanity.com/paper/2503.06457/) (2025)
- [FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning](https://mlsanity.com/paper/2108.06869/) (2021)
- [FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion](https://mlsanity.com/paper/2410.20380/) (2024)
- [Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies](https://mlsanity.com/paper/2109.03999/) (2021)

## Abstract (arXiv)

Federated learning is typically approached as an optimization problem, where the goal is to minimize a global loss function by distributing computation across client devices that possess local data and specify different parts of the global objective. We present an alternative perspective and formulate federated learning as a posterior inference problem, where the goal is to infer a global posterior distribution by having client devices each infer the posterior of their local data. While exact inference is often intractable, this perspective provides a principled way to search for global optima in federated settings. Further, starting with the analysis of federated quadratic objectives, we develop a computation- and communication-efficient approximate posterior inference algorithm -- federated posterior averaging (FedPA). Our algorithm uses MCMC for approximate inference of local posteriors on the clients and efficiently communicates their statistics to the server, where the latter uses them to refine a global estimate of the posterior mode. Finally, we show that FedPA generalizes federated averaging (FedAvg), can similarly benefit from adaptive optimizers, and yields state-of-the-art results on four realistic and challenging benchmarks, converging faster, to better optima.