# Sparse Bayesian Lasso via a Variable-Coefficient ℓ1 Penalty

arXiv: https://arxiv.org/abs/2211.05089 · HTML: https://mlsanity.com/paper/2211.05089/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 61 references that are in the map.
- First posted on arXiv 9 Nov 2022.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [An Alternative Probabilistic Interpretation of the Huber Loss](https://mlsanity.com/paper/1911.02088/) (2019)
- [Proximal Algorithms in Statistics and Machine Learning](https://mlsanity.com/paper/1502.03175/) (2015)
- [Lasso Meets Horseshoe : A Survey](https://mlsanity.com/paper/1706.10179/) (2017)
- [GPU-accelerated Gibbs sampling: a case study of the Horseshoe Probit model](https://mlsanity.com/paper/1608.04329/) (2016)
- [A Bayesian Lasso based Sparse Learning Model](https://mlsanity.com/paper/1908.07220/) (2019)
- [Bayesian Adaptive Lasso with Variational Bayes for Variable Selection in High-dimensional Generalized Linear Mixed Models](https://mlsanity.com/paper/1608.08347/) (2016)
- [Proximal Estimation and Inference](https://mlsanity.com/paper/2205.13469/) (2022)

## Built on by (most cited first, 2 of 2)

- [Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box](https://mlsanity.com/paper/2304.05527/) (2023)
- [VISA: Variational Inference with Sequential Sample-Average Approximations](https://mlsanity.com/paper/2403.09429/) (2024)

## Abstract (arXiv)

Modern statistical learning algorithms are capable of amazing flexibility, but struggle with interpretability. One possible solution is sparsity: making inference such that many of the parameters are estimated as being identically 0, which may be imposed through the use of nonsmooth penalties such as the ℓ1 penalty. However, the ℓ1 penalty introduces significant bias when high sparsity is desired. In this article, we retain the ℓ1 penalty, but define learnable penalty weights λp endowed with hyperpriors. We start the article by investigating the optimization problem this poses, developing a proximal operator associated with the ℓ1 norm. We then study the theoretical properties of this variable-coefficient ℓ1 penalty in the context of penalized likelihood. Next, we investigate application of this penalty to Variational Bayes, developing a model we call the Sparse Bayesian Lasso which allows for behavior qualitatively like Lasso regression to be applied to arbitrary variational models. In simulation studies, this gives us the Uncertainty Quantification and low bias properties of simulation-based approaches with an order of magnitude less computation. Finally, we apply our methodology to a Bayesian lagged spatiotemporal regression model of internal displacement that occurred during the Iraqi Civil War of 2013-2017.