# One-Shot Federated Learning with Bayesian Pseudocoresets

arXiv: https://arxiv.org/abs/2406.02177 · HTML: https://mlsanity.com/paper/2406.02177/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 56 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 30 of 41)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning](https://mlsanity.com/paper/2009.01974/) (2020)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [One-Shot Federated Learning](https://mlsanity.com/paper/1902.11175/) (2019)
- [Privacy for Free: How does Dataset Condensation Help Privacy?](https://mlsanity.com/paper/2206.00240/) (2022)
- [Distilled One-Shot Federated Learning](https://mlsanity.com/paper/2009.07999/) (2020)
- [DENSE: Data-Free One-Shot Federated Learning](https://mlsanity.com/paper/2112.12371/) (2021)
- [Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms](https://mlsanity.com/paper/2010.05273/) (2020)
- [Practical One-Shot Federated Learning for Cross-Silo Setting](https://mlsanity.com/paper/2010.01017/) (2020)
- [FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2207.09653/) (2022)
- [On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes](https://mlsanity.com/paper/1504.07027/) (2015)
- [Federated Learning via Synthetic Data](https://mlsanity.com/paper/2008.04489/) (2020)
- [Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning](https://mlsanity.com/paper/2103.05032/) (2021)
- [Tractable Function-Space Variational Inference in Bayesian Neural Networks](https://mlsanity.com/paper/2312.17199/) (2023)
- [Understanding Variational Inference in Function-Space](https://mlsanity.com/paper/2011.09421/) (2020)
- [FedJAX: Federated learning simulation with JAX](https://mlsanity.com/paper/2108.02117/) (2021)
- [FedSynth: Gradient Compression via Synthetic Data in Federated Learning](https://mlsanity.com/paper/2204.01273/) (2022)
- [Asymptotically Exact, Embarrassingly Parallel MCMC](https://mlsanity.com/paper/1311.4780/) (2013)
- [Partitioned Variational Inference: A unified framework encompassing federated and continual learning](https://mlsanity.com/paper/1811.11206/) (2018)
- [QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning](https://mlsanity.com/paper/2106.00797/) (2021)
- [No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"](https://mlsanity.com/paper/2209.14987/) (2022)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server](https://mlsanity.com/paper/1512.09327/) (2015)
- [Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach](https://mlsanity.com/paper/2302.04228/) (2023)

## Built on by (most cited first, 1 of 1)

- [Federated Martingale Posterior Samping](https://mlsanity.com/paper/2605.18554/) (2026)

## Abstract (arXiv)

Optimization-based techniques for federated learning (FL) often come with prohibitive communication cost, as high dimensional model parameters need to be communicated repeatedly between server and clients. In this paper, we follow a Bayesian approach allowing to perform FL with one-shot communication, by solving the global inference problem as a product of local client posteriors. For models with multi-modal likelihoods, such as neural networks, a naive application of this scheme is hampered, since clients will capture different posterior modes, causing a destructive collapse of the posterior on the server side. Consequently, we explore approximate inference in the function-space representation of client posteriors, hence suffering less or not at all from multi-modality. We show that distributed function-space inference is tightly related to learning Bayesian pseudocoresets and develop a tractable Bayesian FL algorithm on this insight. We show that this approach achieves prediction performance competitive to state-of-the-art while showing a striking reduction in communication cost of up to two orders of magnitude. Moreover, due to its Bayesian nature, our method also delivers well-calibrated uncertainty estimates.