# Scalable Spike-and-Slab

arXiv: https://arxiv.org/abs/2204.01668 · HTML: https://mlsanity.com/paper/2204.01668/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 59 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 11 of 11)

- [Rank-normalization, folding, and localization: An improved widehatR for assessing convergence of MCMC](https://mlsanity.com/paper/1903.08008/) (2019)
- [Variational Bayes for high-dimensional linear regression with sparse priors](https://mlsanity.com/paper/1904.07150/) (2019)
- [Bayesian linear regression with sparse priors](https://mlsanity.com/paper/1403.0735/) (2014)
- [SLOPE - Adaptive variable selection via convex optimization](https://mlsanity.com/paper/1407.3824/) (2014)
- [On the Computational Complexity of High-Dimensional Bayesian Variable Selection](https://mlsanity.com/paper/1505.07925/) (2015)
- [Revisiting the Gelman-Rubin Diagnostic](https://mlsanity.com/paper/1812.09384/) (2018)
- [Lasso Meets Horseshoe : A Survey](https://mlsanity.com/paper/1706.10179/) (2017)
- [Spike and slab variational Bayes for high dimensional logistic regression](https://mlsanity.com/paper/2010.11665/) (2020)
- [Estimating Convergence of Markov chains with L-Lag Couplings](https://mlsanity.com/paper/1905.09971/) (2019)
- [Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO](https://mlsanity.com/paper/2010.06451/) (2020)
- [Coupling-based convergence assessment of some Gibbs samplers for high-dimensional Bayesian regression with shrinkage priors](https://mlsanity.com/paper/2012.04798/) (2020)

## Built on by (most cited first, 5 of 5)

- [Bayesian Concept Bottleneck Models with LLM Priors](https://mlsanity.com/paper/2410.15555/) (2024)
- [Bayesian Variable Selection in a Million Dimensions](https://mlsanity.com/paper/2208.01180/) (2022)
- [Covariate-Elaborated Robust Partial Information Transfer with Conditional Spike-and-Slab Prior](https://mlsanity.com/paper/2404.03764/) (2024)
- [From Estimation to Sampling for Bayesian Linear Regression with Spike-and-Slab Prior](https://mlsanity.com/paper/2307.05558/) (2023)
- [A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression](https://mlsanity.com/paper/2601.21217/) (2026)

## Abstract (arXiv)

Spike-and-slab priors are commonly used for Bayesian variable selection, due to their interpretability and favorable statistical properties. However, existing samplers for spike-and-slab posteriors incur prohibitive computational costs when the number of variables is large. In this article, we propose Scalable Spike-and-Slab (S3), a scalable Gibbs sampling implementation for high-dimensional Bayesian regression with the continuous spike-and-slab prior of George and McCulloch (1993). For a dataset with n observations and p covariates, S3 has order max n2 pt, np  computational cost at iteration t where pt never exceeds the number of covariates switching spike-and-slab states between iterations t and t-1 of the Markov chain. This improves upon the order n2 p per-iteration cost of state-of-the-art implementations as, typically, pt is substantially smaller than p. We apply S3 on synthetic and real-world datasets, demonstrating orders of magnitude speed-ups over existing exact samplers and significant gains in inferential quality over approximate samplers with comparable cost.