Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms
Not on the map. Why?
Abstract (arXiv)
Federated learning is typically approached as an optimization problem, where the goal is to minimize a global loss function by distributing computation across client devices that possess local data and specify different parts of the global objective. We present an alternative perspective and formulate federated learning as a posterior inference problem, where the goal is to infer a global posterior distribution by having client devices each infer the posterior of their local data. While exact inference is often intractable, this perspective provides a principled way to search for global optima in federated settings. Further, starting with the analysis of federated quadratic objectives, we develop a computation- and communication-efficient approximate posterior inference algorithm -- federated posterior averaging (FedPA). Our algorithm uses MCMC for approximate inference of local posteriors on the clients and efficiently communicates their statistics to the server, where the latter uses them to refine a global estimate of the posterior mode. Finally, we show that FedPA generalizes federated averaging (FedAvg), can similarly benefit from adaptive optimizers, and yields state-of-the-art results on four realistic and challenging benchmarks, converging faster, to better optima.
Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 12; 2022: 14; 2023: 21; 2024: 18; 2025: 12; 2026: 1.
78 papers.
Sort:
By area:
- 2021A Field Guide to Federated Optimizationcited by 312
- 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
- 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
- 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
- 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
- 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
- 2021What Do We Mean by Generalization in Federated Learning?cited by 62
- 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
- 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 49
- 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
- 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
38 more
- 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
- 2022Multi-Level Branched Regularization for Federated Learningcited by 34
- 2024Relaxed Contrastive Learning for Federated Learningcited by 32
- 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matterscited by 27
- 2022Communication-Efficient Federated Learning with Accelerated Client Gradientcited by 24
- 2023Bayesian Federated Learning: A Surveycited by 22
- 2023FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregationcited by 22
- 2022SphereFed: Hyperspherical Federated Learningcited by 19
- 2023Federated Learning as Variational Inference: A Scalable Expectation Propagation Approachcited by 17
- 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximizationcited by 17
- 2022Self-Aware Personalized Federated Learningcited by 16
- 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
- 2024Modeling Inter-Intra Heterogeneity for Graph Federated Learningcited by 12
- 2022Federated Learning with Uncertainty via Distilled Predictive Distributionscited by 11
- 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
- 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learningcited by 11
- 2021FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learningcited by 10
- 2021Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategiescited by 9
- 2021An Expectation-Maximization Perspective on Federated Learningcited by 9
- 2022Pisces: Efficient Federated Learning via Guided Asynchronous Trainingcited by 8
- 2023Unimodal Training-Multimodal Prediction: Cross-modal Federated Learning with Hierarchical Aggregationcited by 8
- 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
- 2023Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradientscited by 8
- 2021Federated Learning with Heterogeneous Differential Privacycited by 5
- 2022Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Processcited by 5
- 2022How to Combine Variational Bayesian Networks in Federated Learningcited by 5
- 2023Bayesian Federated Inference for estimating Statistical Models based on Non-shared Multicenter Data setscited by 5
- 2023Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clusteringcited by 5
- 2024Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approachcited by 5
- 2023Federated Variational Inference Methods for Structured Latent Variable Modelscited by 4
- 2023Leveraging Function Space Aggregation for Federated Learning at Scalecited by 4
- 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantificationcited by 4
- 2025Connecting Federated ADMM to Bayescited by 4
- 2023SureFED: Robust Federated Learning via Uncertainty-Aware Inward and Outward Inspectioncited by 3
- 2025Understanding Federated Learning from IID to Non-IID dataset: An Experimental Studycited by 3
- 2025FedBEns: One-Shot Federated Learning based on Bayesian Ensemblecited by 3
- 2025Federated ADMM from Bayesian Dualitycited by 3
- 2023Federated Variational Inference: Towards Improved Personalization and Generalizationcited by 2
50 most cited of 78 All 78 (JSON)
- 2026Certified Uncertainty Propagation in One-Shot Federated Bayesian Models via Posterior Event Transport14 Sep 2026
- 2025Federated Learning via Meta-Variational Dropout23 Oct 2025
- 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning12 Sep 2025
- 2025Calibrating Biased Distribution in VFM-derived Latent Space via Cross-Domain Geometric Consistency19 Aug 2025
- 2025Federated ADMM from Bayesian Duality16 Jun 2025
- 2025Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation20 May 2025
- 2025FedBEns: One-Shot Federated Learning based on Bayesian Ensemble19 Mar 2025
- 2025GC-Fed: Gradient Centralized Federated Learning with Partial Client Participation17 Mar 2025
- 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning9 Mar 2025
- 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework2 Feb 2025
- 2025Understanding Federated Learning from IID to Non-IID dataset: An Experimental Study31 Jan 2025
- 2025Connecting Federated ADMM to Bayes28 Jan 2025
38 more
- 2025Optimizing Privacy-Utility Trade-off in Decentralized Learning with Generalized Correlated Noise24 Jan 2025
- 2024Information-Geometric Barycenters for Bayesian Federated Learning16 Dec 2024
- 2024Modeling Inter-Intra Heterogeneity for Graph Federated Learning16 Dec 2024
- 2024Federated Learning with Uncertainty and Personalization via Efficient Second-order Optimization27 Nov 2024
- 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusion27 Oct 2024
- 2024Distributed Learning for UAV Swarms21 Oct 2024
- 2024Personalizing Low-Rank Bayesian Neural Networks Via Federated Learning18 Oct 2024
- 2024FedPeWS: Personalized Warmup via Subnetworks for Enhanced Heterogeneous Federated Learning3 Oct 2024
- 2024Neighborhood and Global Perturbations Supported SAM in Federated Learning: From Local Tweaks To Global Awareness26 Aug 2024
- 2024Federated Frank-Wolfe Algorithm19 Aug 2024
- 2024FedLog: Personalized Federated Classification with Less Communication and More Flexibility11 Jul 2024
- 2024One-Shot Federated Learning with Bayesian Pseudocoresets4 Jun 2024
- 2024Variational Bayes for Federated Continual Learning23 May 2024
- 2024FedSI: Federated Subnetwork Inference for Efficient Uncertainty Quantification24 Apr 2024
- 2024Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune25 Feb 2024
- 2024Bayesian Federated Inference for regression models based on non-shared multicenter data sets from heterogeneous populations5 Feb 2024
- 2024Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers29 Jan 2024
- 2024Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach16 Jan 2024
- 2024Relaxed Contrastive Learning for Federated Learning10 Jan 2024
- 2023Leveraging Function Space Aggregation for Federated Learning at Scale17 Nov 2023
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark12 Nov 2023
- 2023FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation30 Sep 2023
- 2023Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representations27 Sep 2023
- 2023Distributed Variational Inference for Online Supervised Learning5 Sep 2023
- 2023Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradients8 Aug 2023
- 2023SureFED: Robust Federated Learning via Uncertainty-Aware Inward and Outward Inspection4 Aug 2023
- 2023Personalized Federated Learning via Amortized Bayesian Meta-Learning5 Jul 2023
- 2023Privacy Preserving Bayesian Federated Learning in Heterogeneous Settings13 Jun 2023
- 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matters6 Jun 2023
- 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models29 May 2023
- 2023Federated Variational Inference: Towards Improved Personalization and Generalization23 May 2023
- 2023Confidence-aware Personalized Federated Learning via Variational Expectation Maximization21 May 2023
- 2023FedHB: Hierarchical Bayesian Federated Learning8 May 2023
- 2023Bayesian Federated Learning: A Survey26 Apr 2023
- 2023Unimodal Training-Multimodal Prediction: Cross-modal Federated Learning with Hierarchical Aggregation27 Mar 2023
- 2023Comparative Evaluation of Data Decoupling Techniques for Federated Machine Learning with Database as a Service15 Mar 2023
- 2023Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering8 Mar 2023
- 2023Bayesian Federated Inference for estimating Statistical Models based on Non-shared Multicenter Data sets15 Feb 2023
50 newest of 78 All 78 (JSON)
- 2021A Field Guide to Federated Optimizationcited by 312
- 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
- 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
- 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
- 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
- 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
- 2021What Do We Mean by Generalization in Federated Learning?cited by 62
- 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
- 2022FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Datacited by 49
- 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
- 2021A Bayesian Federated Learning Framework with Online Laplace Approximationcited by 41
- 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
12 most cited of 77 All 77 (JSON)
- 2026Certified Uncertainty Propagation in One-Shot Federated Bayesian Models via Posterior Event Transport14 Sep 2026
- 2025Federated Learning via Meta-Variational Dropout23 Oct 2025
- 2025Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning12 Sep 2025
- 2025Calibrating Biased Distribution in VFM-derived Latent Space via Cross-Domain Geometric Consistency19 Aug 2025
- 2025Federated ADMM from Bayesian Duality16 Jun 2025
- 2025Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation20 May 2025
- 2025FedBEns: One-Shot Federated Learning based on Bayesian Ensemble19 Mar 2025
- 2025GC-Fed: Gradient Centralized Federated Learning with Partial Client Participation17 Mar 2025
- 2025Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning9 Mar 2025
- 2025Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework2 Feb 2025
- 2025Understanding Federated Learning from IID to Non-IID dataset: An Experimental Study31 Jan 2025
- 2025Connecting Federated ADMM to Bayes28 Jan 2025
12 newest of 77 All 77 (JSON)