# Partitioned Variational Inference: A unified framework encompassing federated and continual learning

arXiv: https://arxiv.org/abs/1811.11206 · HTML: https://mlsanity.com/paper/1811.11206/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2018.

## Built on by (most cited first, 29 of 29)

- [Personalized Federated Learning with Gaussian Processes](https://mlsanity.com/paper/2106.15482/) (2021)
- [TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation](https://mlsanity.com/paper/2303.06937/) (2023)
- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [Communication Efficiency in Federated Learning: Achievements and Challenges](https://mlsanity.com/paper/2107.10996/) (2021)
- [FedPop: A Bayesian Approach for Personalised Federated Learning](https://mlsanity.com/paper/2206.03611/) (2022)
- [QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning](https://mlsanity.com/paper/2106.00797/) (2021)
- [Towards Federated Learning on Time-Evolving Heterogeneous Data](https://mlsanity.com/paper/2112.13246/) (2021)
- [Forget-SVGD: Particle-Based Bayesian Federated Unlearning](https://mlsanity.com/paper/2111.12056/) (2021)
- [Text-Enhanced Data-free Approach for Federated Class-Incremental Learning](https://mlsanity.com/paper/2403.14101/) (2024)
- [Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees](https://mlsanity.com/paper/2111.01721/) (2021)
- [DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs](https://mlsanity.com/paper/2106.06300/) (2021)
- [Parallel MCMC Without Embarrassing Failures](https://mlsanity.com/paper/2202.11154/) (2022)
- [Bayesian Variational Federated Learning and Unlearning in Decentralized Networks](https://mlsanity.com/paper/2104.03834/) (2021)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [Precision-Weighted Federated Learning](https://mlsanity.com/paper/2107.09627/) (2021)
- [Compressed Particle-Based Federated Bayesian Learning and Unlearning](https://mlsanity.com/paper/2209.07267/) (2022)
- [Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach](https://mlsanity.com/paper/2401.08351/) (2024)
- [Sparse Gaussian Process Hyperparameters: Optimize or Integrate?](https://mlsanity.com/paper/2211.02476/) (2022)
- [Client Selection for Federated Bayesian Learning](https://mlsanity.com/paper/2212.05492/) (2022)
- [Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations](https://mlsanity.com/paper/2408.06069/) (2024)
- [Connecting Federated ADMM to Bayes](https://mlsanity.com/paper/2501.17325/) (2025)
- [Differentially private partitioned variational inference](https://mlsanity.com/paper/2209.11595/) (2022)
- [Modular Gaussian Processes for Transfer Learning](https://mlsanity.com/paper/2110.13515/) (2021)
- [Distributed Variational Inference for Online Supervised Learning](https://mlsanity.com/paper/2309.02606/) (2023)
- [One-Shot Federated Learning with Bayesian Pseudocoresets](https://mlsanity.com/paper/2406.02177/) (2024)
- [Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework](https://mlsanity.com/paper/2502.00846/) (2025)
- [Robust Distributed Bayesian Learning with Stragglers via Consensus Monte Carlo](https://mlsanity.com/paper/2112.09794/) (2021)
- [Fearless Stochasticity in Expectation Propagation](https://mlsanity.com/paper/2406.01801/) (2024)
- [Neural composite likelihood estimation: simulation based inference for time series](https://mlsanity.com/paper/2609.21762/) (2026)

## Abstract (arXiv)

Variational inference (VI) has become the method of choice for fitting many modern probabilistic models. However, practitioners are faced with a fragmented literature that offers a bewildering array of algorithmic options. First, the variational family. Second, the granularity of the updates e.g. whether the updates are local to each data point and employ message passing or global. Third, the method of optimization (bespoke or blackbox, closed-form or stochastic updates, etc.). This paper presents a new framework, termed Partitioned Variational Inference (PVI), that explicitly acknowledges these algorithmic dimensions of VI, unifies disparate literature, and provides guidance on usage. Crucially, the proposed PVI framework allows us to identify new ways of performing VI that are ideally suited to challenging learning scenarios including federated learning (where distributed computing is leveraged to process non-centralized data) and continual learning (where new data and tasks arrive over time and must be accommodated quickly). We showcase these new capabilities by developing communication-efficient federated training of Bayesian neural networks and continual learning for Gaussian process models with private pseudo-points. The new methods significantly outperform the state-of-the-art, whilst being almost as straightforward to implement as standard VI.