# Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem

arXiv: https://arxiv.org/abs/1601.06207 · HTML: https://mlsanity.com/paper/1601.06207/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2016.

## Built on by (most cited first, 2 of 2)

- [Covariance-Free Sparse Bayesian Learning](https://mlsanity.com/paper/2105.10439/) (2021)
- [High-Dimensional Sparse Bayesian Learning without Covariance Matrices](https://mlsanity.com/paper/2202.12808/) (2022)

## Abstract (arXiv)

In this paper, we develop a Bayesian evidence maximization framework to solve the sparse non-negative least squares (S-NNLS) problem. We introduce a family of probability densities referred to as the Rectified Gaussian Scale Mixture (R- GSM) to model the sparsity enforcing prior distribution for the solution. The R-GSM prior encompasses a variety of heavy-tailed densities such as the rectified Laplacian and rectified Student- t distributions with a proper choice of the mixing density. We utilize the hierarchical representation induced by the R-GSM prior and develop an evidence maximization framework based on the Expectation-Maximization (EM) algorithm. Using the EM based method, we estimate the hyper-parameters and obtain a point estimate for the solution. We refer to the proposed method as rectified sparse Bayesian learning (R-SBL). We provide four R- SBL variants that offer a range of options for computational complexity and the quality of the E-step computation. These methods include the Markov chain Monte Carlo EM, linear minimum mean-square-error estimation, approximate message passing and a diagonal approximation. Using numerical experiments, we show that the proposed R-SBL method outperforms existing S-NNLS solvers in terms of both signal and support recovery performance, and is also very robust against the structure of the design matrix.