# High-Dimensional Sparse Bayesian Learning without Covariance Matrices

arXiv: https://arxiv.org/abs/2202.12808 · HTML: https://mlsanity.com/paper/2202.12808/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 25 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 5 of 5)

- [Variational Relevance Vector Machines](https://mlsanity.com/paper/1301.3838/) (2013)
- [Pattern-Coupled Sparse Bayesian Learning for Recovery of Block-Sparse Signals](https://mlsanity.com/paper/1311.2150/) (2013)
- [A GAMP Based Low Complexity Sparse Bayesian Learning Algorithm](https://mlsanity.com/paper/1703.03044/) (2017)
- [A Bayesian Framework for Sparse Representation-Based 3D Human Pose Estimation](https://mlsanity.com/paper/1412.0062/) (2014)
- [Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem](https://mlsanity.com/paper/1601.06207/) (2016)

## Built on by (most cited first, 2 of 2)

- [Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models](https://mlsanity.com/paper/2306.03249/) (2023)
- [An Efficient Algorithm for Clustered Multi-Task Compressive Sensing](https://mlsanity.com/paper/2310.00420/) (2023)

## Abstract (arXiv)

Sparse Bayesian learning (SBL) is a powerful framework for tackling the sparse coding problem. However, the most popular inference algorithms for SBL become too expensive for high-dimensional settings, due to the need to store and compute a large covariance matrix. We introduce a new inference scheme that avoids explicit construction of the covariance matrix by solving multiple linear systems in parallel to obtain the posterior moments for SBL. Our approach couples a little-known diagonal estimation result from numerical linear algebra with the conjugate gradient algorithm. On several simulations, our method scales better than existing approaches in computation time and memory, especially for structured dictionaries capable of fast matrix-vector multiplication.