# Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm

arXiv: https://arxiv.org/abs/2209.08139 · HTML: https://mlsanity.com/paper/2209.08139/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 111 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 14 of 14)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Variational Bayes for high-dimensional linear regression with sparse priors](https://mlsanity.com/paper/1904.07150/) (2019)
- [Bayesian linear regression with sparse priors](https://mlsanity.com/paper/1403.0735/) (2014)
- [Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data](https://mlsanity.com/paper/1412.4869/) (2014)
- [Empirical Bayes posterior concentration in sparse high-dimensional linear models](https://mlsanity.com/paper/1406.7718/) (2014)
- [A flexible empirical Bayes approach to multiple linear regression and connections with penalized regression](https://mlsanity.com/paper/2208.10910/) (2022)
- [A Variational Algorithm for Bayesian Variable Selection](https://mlsanity.com/paper/1602.07640/) (2016)
- [Empirical priors for prediction in sparse high-dimensional linear regression](https://mlsanity.com/paper/1903.00961/) (2019)
- [An Ensemble EM Algorithm for Bayesian Variable Selection](https://mlsanity.com/paper/1603.04360/) (2016)
- [Methods for Bayesian Variable Selection with Binary Response Data using the EM Algorithm](https://mlsanity.com/paper/1605.05429/) (2016)
- [An Expectation Conditional Maximization approach for Gaussian graphical models](https://mlsanity.com/paper/1709.06970/) (2017)
- [Honest confidence sets for high-dimensional regression by projection and shrinkage](https://mlsanity.com/paper/1902.00535/) (2019)

## Built on by (most cited first, 2 of 2)

- [Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression](https://mlsanity.com/paper/2309.08783/) (2023)
- [Sparse high-dimensional linear mixed modeling with a partitioned empirical Bayes ECM algorithm](https://mlsanity.com/paper/2310.12285/) (2023)

## Abstract (arXiv)

Bayesian variable selection methods are powerful techniques for fitting and inferring on sparse high-dimensional linear regression models. However, many are computationally intensive or require restrictive prior distributions on model parameters. In this paper, we proposed a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are required through the use of plug-in empirical Bayes estimates of hyperparameters. Efficient maximum a posteriori (MAP) estimation is completed through a Parameter-Expanded Expectation-Conditional-Maximization (PX-ECM) algorithm. The PX-ECM results in a robust computationally efficient coordinate-wise optimization which -- when updating the coefficient for a particular predictor -- adjusts for the impact of other predictor variables. The completion of the E-step uses an approach motivated by the popular two-group approach to multiple testing. The result is a PaRtitiOned empirical Bayes Ecm (PROBE) algorithm applied to sparse high-dimensional linear regression, which can be completed using one-at-a-time or all-at-once type optimization. We compare the empirical properties of PROBE to comparable approaches with numerous simulation studies and analyses of cancer cell drug responses. The proposed approach is implemented in the R package probe.