# An Ensemble EM Algorithm for Bayesian Variable Selection

arXiv: https://arxiv.org/abs/1603.04360 · HTML: https://mlsanity.com/paper/1603.04360/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2016.

## Built on by (most cited first, 2 of 2)

- [Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm](https://mlsanity.com/paper/2209.08139/) (2022)
- [On the optimization and pruning for Bayesian deep learning](https://mlsanity.com/paper/2210.12957/) (2022)

## Abstract (arXiv)

We study the Bayesian approach to variable selection in the context of linear regression. Motivated by a recent work by Rockova and George (2014), we propose an EM algorithm that returns the MAP estimate of the set of relevant variables. Due to its particular updating scheme, our algorithm can be implemented efficiently without inverting a large matrix in each iteration and therefore can scale up with big data. We also show that the MAP estimate returned by our EM algorithm achieves variable selection consistency even when p diverges with n. In practice, our algorithm could get stuck with local modes, a common problem with EM algorithms. To address this issue, we propose an ensemble EM algorithm, in which we repeatedly apply the EM algorithm on a subset of the samples with a subset of the covariates, and then aggregate the variable selection results across those bootstrap replicates. Empirical studies have demonstrated the superior performance of the ensemble EM algorithm.