# Variational empirical Bayes variable selection in high-dimensional logistic regression

arXiv: https://arxiv.org/abs/2502.10532 · HTML: https://mlsanity.com/paper/2502.10532/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 14 Feb 2025.

## Builds on (most cited first, 18 of 18)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Concentration of tempered posteriors and of their variational approximations](https://mlsanity.com/paper/1706.09293/) (2017)
- [Convergence Rates of Variational Posterior Distributions](https://mlsanity.com/paper/1712.02519/) (2017)
- [Variational Bayes for high-dimensional linear regression with sparse priors](https://mlsanity.com/paper/1904.07150/) (2019)
- [Sparsity information and regularization in the horseshoe and other shrinkage priors](https://mlsanity.com/paper/1707.01694/) (2017)
- [The Impact of Regularization on High-dimensional Logistic Regression](https://mlsanity.com/paper/1906.03761/) (2019)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Robust Bayesian inference via coarsening](https://mlsanity.com/paper/1506.06101/) (2015)
- [Calibrating general posterior credible regions](https://mlsanity.com/paper/1509.00922/) (2015)
- [Adaptive variational Bayes: Optimality, computation and applications](https://mlsanity.com/paper/2109.03204/) (2021)
- [Spike and slab variational Bayes for high dimensional logistic regression](https://mlsanity.com/paper/2010.11665/) (2020)
- [Empirical Bayes posterior concentration in sparse high-dimensional linear models](https://mlsanity.com/paper/1406.7718/) (2014)
- [A Variational Algorithm for Bayesian Variable Selection](https://mlsanity.com/paper/1602.07640/) (2016)
- [Variational approximations of empirical Bayes posteriors in high-dimensional linear models](https://mlsanity.com/paper/2007.15930/) (2020)
- [Empirical Bayes inference in sparse high-dimensional generalized linear models](https://mlsanity.com/paper/2303.07854/) (2023)
- [Methods for Bayesian Variable Selection with Binary Response Data using the EM Algorithm](https://mlsanity.com/paper/1605.05429/) (2016)
- [A Variational Inference method for Bayesian variable selection](https://mlsanity.com/paper/2211.11383/) (2022)
- [An empirical G-Wishart prior for sparse high-dimensional Gaussian graphical models](https://mlsanity.com/paper/1912.03807/) (2019)

## Abstract (arXiv)

Logistic regression involving high-dimensional covariates is a practically important problem. Often the goal is variable selection, i.e., determining which few of the many covariates are associated with the binary response. Unfortunately, the usual Bayesian computations can be quite challenging and expensive. Here we start with a recently proposed empirical Bayes solution, with strong theoretical convergence properties, and develop a novel and computationally efficient variational approximation thereof. One such novelty is that we develop this approximation directly for the marginal distribution on the model space, rather than on the regression coefficients themselves. We demonstrate the method's strong performance in simulations, and prove that our variational approximation inherits the strong selection consistency property satisfied by the posterior distribution that it is approximating.