# Merging MCMC Subposteriors through Gaussian-Process Approximations

arXiv: https://arxiv.org/abs/1605.08576 · HTML: https://mlsanity.com/paper/1605.08576/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2016.

## Built on by (most cited first, 13 of 13)

- [QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning](https://mlsanity.com/paper/2106.00797/) (2021)
- [DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs](https://mlsanity.com/paper/2106.06300/) (2021)
- [Parallel MCMC Without Embarrassing Failures](https://mlsanity.com/paper/2202.11154/) (2022)
- [SwISS: A Scalable Markov chain Monte Carlo Divide-and-Conquer Strategy](https://mlsanity.com/paper/2208.04080/) (2022)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Divide-and-Conquer Fusion](https://mlsanity.com/paper/2110.07265/) (2021)
- [Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature](https://mlsanity.com/paper/2303.05263/) (2023)
- [Simulation-based stacking](https://mlsanity.com/paper/2310.17009/) (2023)
- [Embarrassingly Parallel GFlowNets](https://mlsanity.com/paper/2406.03288/) (2024)
- [Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms](https://mlsanity.com/paper/2211.00100/) (2022)
- [Diffusion Generative Modelling for Divide-and-Conquer MCMC](https://mlsanity.com/paper/2406.11664/) (2024)
- [Measuring the right thing: justifying metrics in AI impact assessments](https://mlsanity.com/paper/2504.05007/) (2025)
- [Sampling as Bandits: Evaluation-Efficient Design for Black-Box Densities](https://mlsanity.com/paper/2509.01437/) (2025)

## Abstract (arXiv)

Markov chain Monte Carlo (MCMC) algorithms have become powerful tools for Bayesian inference. However, they do not scale well to large-data problems. Divide-and-conquer strategies, which split the data into batches and, for each batch, run independent MCMC algorithms targeting the corresponding subposterior, can spread the computational burden across a number of separate workers. The challenge with such strategies is in recombining the subposteriors to approximate the full posterior. By creating a Gaussian-process approximation for each log-subposterior density we create a tractable approximation for the full posterior. This approximation is exploited through three methodologies: firstly a Hamiltonian Monte Carlo algorithm targeting the expectation of the posterior density provides a sample from an approximation to the posterior; secondly, evaluating the true posterior at the sampled points leads to an importance sampler that, asymptotically, targets the true posterior expectations; finally, an alternative importance sampler uses the full Gaussian-process distribution of the approximation to the log-posterior density to re-weight any initial sample and provide both an estimate of the posterior expectation and a measure of the uncertainty in it.