# Scaling up Data Augmentation MCMC via Calibration

arXiv: https://arxiv.org/abs/1703.03123 · HTML: https://mlsanity.com/paper/1703.03123/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2017.

## Built on by (most cited first, 4 of 4)

- [Bayesian neural networks via MCMC: a Python-based tutorial](https://mlsanity.com/paper/2304.02595/) (2023)
- [A Scalable and Efficient Iterative Method for Copying Machine Learning Classifiers](https://mlsanity.com/paper/2302.02667/) (2023)
- [Nonparametric posterior learning for emission tomography with multimodal data](https://mlsanity.com/paper/2108.00866/) (2021)
- [The data augmentation algorithm](https://mlsanity.com/paper/2406.10464/) (2024)

## Abstract (arXiv)

There has been considerable interest in making Bayesian inference more scalable. In big data settings, most literature focuses on reducing the computing time per iteration, with less focused on reducing the number of iterations needed in Markov chain Monte Carlo (MCMC). This article focuses on data augmentation MCMC (DA-MCMC), a widely used technique. DA-MCMC samples tend to become highly autocorrelated in large data samples, due to a miscalibration problem in which conditional posterior distributions given augmented data are too concentrated. This makes it necessary to collect very long MCMC paths to obtain acceptably low MC error. To combat this inefficiency, we propose a family of calibrated data augmentation algorithms, which appropriately adjust the variance of conditional posterior distributions. A Metropolis-Hastings step is used to eliminate bias in the stationary distribution of the resulting sampler. Compared to existing alternatives, this approach can dramatically reduce MC error by reducing autocorrelation and increasing the effective number of DA-MCMC samples per computing time. The approach is simple and applicable to a broad variety of existing data augmentation algorithms, and we focus on three popular models: probit, logistic and Poisson log-linear. Dramatic gains in computational efficiency are shown in applications.