mlsanityData to 7 Oct 2026

Variational Bayes for high-dimensional linear regression with sparse priors

arXiv 1904.07150 ↗First posted 15 Apr 2019stat.ME math.ST stat.MLcited by 44

Not on the map. Why?

Abstract (arXiv)

We study a mean-field spike and slab variational Bayes (VB) approximation to Bayesian model selection priors in sparse high-dimensional linear regression. Under compatibility conditions on the design matrix, oracle inequalities are derived for the mean-field VB approximation, implying that it converges to the sparse truth at the optimal rate and gives optimal prediction of the response vector. The empirical performance of our algorithm is studied, showing that it works comparably well as other state-of-the-art Bayesian variable selection methods. We also numerically demonstrate that the widely used coordinate-ascent variational inference (CAVI) algorithm can be highly sensitive to the parameter updating order, leading to potentially poor performance. To mitigate this, we propose a novel prioritized updating scheme that uses a data-driven updating order and performs better in simulations. The variational algorithm is implemented in the R package 'sparsevb'.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 9; 2023: 10; 2024: 7; 2025: 6; 2026: 4.

44 papers.

Sort:

  1. 2023On the Convergence of Coordinate Ascent Variational Inferencecited by 18
  2. 2021Adaptive variational Bayes: Optimality, computation and applicationscited by 17
  3. 2022Thompson Sampling for High-Dimensional Sparse Linear Contextual Banditscited by 16
  4. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
  5. 2021Variational Inference in high-dimensional linear regressioncited by 10
  6. 2022The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?cited by 9
  7. 2022A flexible empirical Bayes approach to multiple linear regression and connections with penalized regressioncited by 8
  8. 2022On the inability of Gaussian process regression to optimally learn compositional functionscited by 7
  9. 2022Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selectioncited by 7
  10. 2021Performance of Bayesian linear regression in a model with mismatchcited by 6
  11. 2023Additive Multi-Index Gaussian process modeling, with application to multi-physics surrogate modeling of the quark-gluon plasmacited by 6
  12. 2024A sparse PAC-Bayesian approach for high-dimensional quantile predictioncited by 6
32 more
  1. 2022Scalable Spike-and-Slabcited by 5
  2. 2023Machine Learning and the Future of Bayesian Computationcited by 4
  3. 2023Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networkscited by 4
  4. 2023A Mean Field Approach to Empirical Bayes Estimation in High-dimensional Linear Regressioncited by 4
  5. 2021Spike and slab Bayesian sparse principal component analysiscited by 3
  6. 2021A fast asynchronous MCMC sampler for sparse Bayesian inferencecited by 3
  7. 2021Variational Bayes for high-dimensional proportional hazards models with applications within gene expressioncited by 3
  8. 2021Sparse online variational Bayesian regressioncited by 2
  9. 2022Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithmcited by 2
  10. 2023Mixed Variational Flows for Discrete Variablescited by 2
  11. 2023Variational Gaussian Processes For Linear Inverse Problemscited by 2
  12. 2024Covariate-Elaborated Robust Partial Information Transfer with Conditional Spike-and-Slab Priorcited by 2
  13. 2024Contraction rates for conjugate gradient and Lanczos approximate posteriors in Gaussian process regressioncited by 2
  14. 2025Adaptive sparse variational approximations for Gaussian process regressioncited by 2
  15. 2023Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion priorcited by 1
  16. 2024Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variancecited by 1
  17. 2025Robust and Scalable Variational Bayescited by 1
  18. 2026Stability of Sequential and Parallel Coordinate Ascent Variational Inferencecited by 1
  19. 2022Factorized Fusion Shrinkage for Dynamic Relational Datacited by 0
  20. 2022Scalable and adaptive variational Bayes methods for Hawkes processescited by 0
  21. 2023From Estimation to Sampling for Bayesian Linear Regression with Spike-and-Slab Priorcited by 0
  22. 2023The surrogate Gibbs-posterior of a corrected stochastic MALA: Towards uncertainty quantification for neural networkscited by 0
  23. 2024High-dimensional prediction for count response via sparse exponential weightscited by 0
  24. 2024Variational Bayes Portfolio Constructioncited by 0
  25. 2024Variational Bayesian Bow tie Neural Networks with Shrinkagecited by 0
  26. 2025Variational empirical Bayes variable selection in high-dimensional logistic regressioncited by 0
  27. 2025EB-gMCR: Energy-Based Generative Modeling for Signal Unmixing and Multivariate Curve Resolutioncited by 0
  28. 2025BaGGLS: A Bayesian Shrinkage Framework for Interpretable Modeling of Interactions in High-Dimensional Biological Datacited by 0
  29. 2025Variational bagging: a robust approach for Bayesian uncertainty quantificationcited by 0
  30. 2026Variational predictive resamplingcited by 0
  31. 2026Joint Model and Data Sparsification via the Marginal Likelihoodcited by 0
  32. 2026Adaptive mixture variational inference for spike-and-slab regressioncited by 0