# Scalability of Metropolis-within-Gibbs schemes for high-dimensional Bayesian models

arXiv: https://arxiv.org/abs/2403.09416 · HTML: https://mlsanity.com/paper/2403.09416/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 82 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 13 of 13)

- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015)
- [Log-concave sampling: Metropolis-Hastings algorithms are fast](https://mlsanity.com/paper/1801.02309/) (2018)
- [Minimax Mixing Time of the Metropolis-Adjusted Langevin Algorithm for Log-Concave Sampling](https://mlsanity.com/paper/2109.13055/) (2021)
- [The Barker proposal: combining robustness and efficiency in gradient-based MCMC](https://mlsanity.com/paper/1908.11812/) (2019)
- [Estimating Convergence of Markov chains with L-Lag Couplings](https://mlsanity.com/paper/1905.09971/) (2019)
- [Scan Order in Gibbs Sampling: Models in Which it Matters and Bounds on How Much](https://mlsanity.com/paper/1606.03432/) (2016)
- [Scalable Bayesian computation for crossed and nested hierarchical models](https://mlsanity.com/paper/2103.10875/) (2021)
- [Ancillarity-Sufficiency Interweaving Strategy (ASIS) for Boosting MCMC Estimation of Stochastic Volatility Models](https://mlsanity.com/paper/1706.05280/) (2017)
- [Scalable inference for crossed random effects models](https://mlsanity.com/paper/1803.09460/) (2018)
- [MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure](https://mlsanity.com/paper/1908.09429/) (2019)
- [Dimension-free mixing times of Gibbs samplers for Bayesian hierarchical models](https://mlsanity.com/paper/2304.06993/) (2023)
- [Spectral Telescope: Convergence Rate Bounds for Random-Scan Gibbs Samplers Based on a Hierarchical Structure](https://mlsanity.com/paper/2208.11299/) (2022)

## Built on by (most cited first, 4 of 4)

- [Entropy contraction of the Gibbs sampler under log-concavity](https://mlsanity.com/paper/2410.00858/) (2024)
- [Dimension-free Relaxation Times of Informed MCMC Samplers on Discrete Spaces](https://mlsanity.com/paper/2404.03867/) (2024)
- [Spectral gap of Metropolis-within-Gibbs under log-concavity](https://mlsanity.com/paper/2509.26175/) (2025)
- [Regularized Large Neighborhood Search](https://mlsanity.com/paper/2606.02294/) (2026)

## Abstract (arXiv)

We study general coordinate-wise MCMC schemes (such as Metropolis-within-Gibbs samplers), which are commonly used to fit Bayesian non-conjugate hierarchical models. We relate their convergence properties to the ones of the corresponding (potentially not implementable) Gibbs sampler through the notion of conditional conductance. This allows us to study the performances of popular Metropolis-within-Gibbs schemes for non-conjugate hierarchical models, in high-dimensional regimes where both number of datapoints and parameters increase. Given random data-generating assumptions, we establish dimension-free convergence results, which are in close accordance with numerical evidences. Applications to Bayesian models for binary regression with unknown hyperparameters and discretely observed diffusions are also discussed. Motivated by such statistical applications, auxiliary results of independent interest on approximate conductances and perturbation of Markov operators are provided.