# Bayesian linear regression with sparse priors

arXiv: https://arxiv.org/abs/1403.0735 · HTML: https://mlsanity.com/paper/1403.0735/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2014.

## Built on by (most cited first, 30 of 40)

- [Identifiable Deep Generative Models via Sparse Decoding](https://mlsanity.com/paper/2110.10804/) (2021)
- [Bayesian Concept Bottleneck Models with LLM Priors](https://mlsanity.com/paper/2410.15555/) (2024)
- [Thompson Sampling for High-Dimensional Sparse Linear Contextual Bandits](https://mlsanity.com/paper/2211.05964/) (2022)
- [Variational Inference in high-dimensional linear regression](https://mlsanity.com/paper/2104.12232/) (2021)
- [The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?](https://mlsanity.com/paper/2205.10009/) (2022)
- [Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inference](https://mlsanity.com/paper/2305.14765/) (2023)
- [Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes](https://mlsanity.com/paper/2111.04558/) (2021)
- [Asymptotic Properties for Bayesian Neural Network in Besov Space](https://mlsanity.com/paper/2206.00241/) (2022)
- [Misclassification bounds for PAC-Bayesian sparse deep learning](https://mlsanity.com/paper/2405.01304/) (2024)
- [Performance of Bayesian linear regression in a model with mismatch](https://mlsanity.com/paper/2107.06936/) (2021)
- [A sparse PAC-Bayesian approach for high-dimensional quantile prediction](https://mlsanity.com/paper/2409.01687/) (2024)
- [Scalable Spike-and-Slab](https://mlsanity.com/paper/2204.01668/) (2022)
- [Robust Methods for High-Dimensional Linear Learning](https://mlsanity.com/paper/2208.05447/) (2022)
- [Bayesian Joint Additive Factor Models for Multiview Learning](https://mlsanity.com/paper/2406.00778/) (2024)
- [FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout](https://mlsanity.com/paper/2307.07172/) (2023)
- [High-dimensional Bayesian Tobit regression for censored response with Horseshoe prior](https://mlsanity.com/paper/2505.08288/) (2025)
- [Spike and slab Bayesian sparse principal component analysis](https://mlsanity.com/paper/2102.00305/) (2021)
- [A fast asynchronous MCMC sampler for sparse Bayesian inference](https://mlsanity.com/paper/2108.06446/) (2021)
- [Hierarchical shrinkage Gaussian processes: applications to computer code emulation and dynamical system recovery](https://mlsanity.com/paper/2302.00755/) (2023)
- [Sparse online variational Bayesian regression](https://mlsanity.com/paper/2102.12261/) (2021)
- [Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm](https://mlsanity.com/paper/2209.08139/) (2022)
- [On the optimization and pruning for Bayesian deep learning](https://mlsanity.com/paper/2210.12957/) (2022)
- [Covariate-Elaborated Robust Partial Information Transfer with Conditional Spike-and-Slab Prior](https://mlsanity.com/paper/2404.03764/) (2024)
- [Handling bounded response in high dimensions: a Horseshoe prior Bayesian Beta regression approach](https://mlsanity.com/paper/2505.22211/) (2025)
- [Masked Bayesian Neural Networks : Computation and Optimality](https://mlsanity.com/paper/2206.00853/) (2022)
- [Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance](https://mlsanity.com/paper/2405.19016/) (2024)
- [Isoperimetry is All We Need: Langevin Posterior Sampling for RL with Sublinear Regret](https://mlsanity.com/paper/2412.20824/) (2024)
- [Correcting Noisy Multilabel Predictions: Modeling Label Noise through Latent Space Shifts](https://mlsanity.com/paper/2502.14281/) (2025)
- [Spectral Bayesian Regression on the Sphere](https://mlsanity.com/paper/2601.20528/) (2026)
- [Stability of Sequential and Parallel Coordinate Ascent Variational Inference](https://mlsanity.com/paper/2603.20929/) (2026)

## Abstract (arXiv)

We study full Bayesian procedures for high-dimensional linear regression under sparsity constraints. The prior is a mixture of point masses at zero and continuous distributions. Under compatibility conditions on the design matrix, the posterior distribution is shown to contract at the optimal rate for recovery of the unknown sparse vector, and to give optimal prediction of the response vector. It is also shown to select the correct sparse model, or at least the coefficients that are significantly different from zero. The asymptotic shape of the posterior distribution is characterized and employed to the construction and study of credible sets for uncertainty quantification.