mlsanityData to 14 Oct 2026

Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm

arXiv 2209.08139 ↗First posted 16 Sep 2022stat.ME stat.MLcited by 2

Not on the map. Why?

Abstract (arXiv)

Bayesian variable selection methods are powerful techniques for fitting and inferring on sparse high-dimensional linear regression models. However, many are computationally intensive or require restrictive prior distributions on model parameters. In this paper, we proposed a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are required through the use of plug-in empirical Bayes estimates of hyperparameters. Efficient maximum a posteriori (MAP) estimation is completed through a Parameter-Expanded Expectation-Conditional-Maximization (PX-ECM) algorithm. The PX-ECM results in a robust computationally efficient coordinate-wise optimization which -- when updating the coefficient for a particular predictor -- adjusts for the impact of other predictor variables. The completion of the E-step uses an approach motivated by the popular two-group approach to multiple testing. The result is a PaRtitiOned empirical Bayes Ecm (PROBE) algorithm applied to sparse high-dimensional linear regression, which can be completed using one-at-a-time or all-at-once type optimization. We compare the empirical properties of PROBE to comparable approaches with numerous simulation studies and analyses of cancer cell drug responses. The proposed approach is implemented in the R package probe.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 14 of its 111 references that are in the map. First posted on arXiv 16 Sep 2022.

Builds on

14 of 111 references.

References · 14

  1. 2016Variational Inference: A Review for Statisticianscited by 1,472
  2. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  3. 2019Conformal Prediction Under Covariate Shiftcited by 519
  4. 2019Variational Bayes for high-dimensional linear regression with sparse priorscited by 44
  5. 2014Bayesian linear regression with sparse priorscited by 40
  6. 2014Expectation propagation as a way of life: A framework for Bayesian inference on partitioned datacited by 14
  7. 2014Empirical Bayes posterior concentration in sparse high-dimensional linear modelscited by 10
  8. 2022A flexible empirical Bayes approach to multiple linear regression and connections with penalized regressioncited by 8
  9. 2016A Variational Algorithm for Bayesian Variable Selectioncited by 6
  10. 2019Empirical priors for prediction in sparse high-dimensional linear regressioncited by 6
  11. 2016An Ensemble EM Algorithm for Bayesian Variable Selectioncited by 2
  12. 2016Methods for Bayesian Variable Selection with Binary Response Data using the EM Algorithmcited by 2
2 more
  1. 2017An Expectation Conditional Maximization approach for Gaussian graphical modelscited by 2
  2. 2019Honest confidence sets for high-dimensional regression by projection and shrinkagecited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regressioncited by 0
  2. 2023Sparse high-dimensional linear mixed modeling with a partitioned empirical Bayes ECM algorithmcited by 0