# Covariance-Free Sparse Bayesian Learning

arXiv: https://arxiv.org/abs/2105.10439 · HTML: https://mlsanity.com/paper/2105.10439/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 63 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 7 of 7)

- [Variational Relevance Vector Machines](https://mlsanity.com/paper/1301.3838/) (2013)
- [Pattern-Coupled Sparse Bayesian Learning for Recovery of Block-Sparse Signals](https://mlsanity.com/paper/1311.2150/) (2013)
- [Fast Online Deconvolution of Calcium Imaging Data](https://mlsanity.com/paper/1609.00639/) (2016)
- [A GAMP Based Low Complexity Sparse Bayesian Learning Algorithm](https://mlsanity.com/paper/1703.03044/) (2017)
- [A Bayesian Framework for Sparse Representation-Based 3D Human Pose Estimation](https://mlsanity.com/paper/1412.0062/) (2014)
- [Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem](https://mlsanity.com/paper/1601.06207/) (2016)
- [Off-Grid DOA Estimation Using Sparse Bayesian Learning in MIMO Radar With Unknown Mutual Coupling](https://mlsanity.com/paper/1804.04460/) (2018)

## Built on by (most cited first, 2 of 2)

- [Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models](https://mlsanity.com/paper/2306.03249/) (2023)
- [An Efficient Algorithm for Clustered Multi-Task Compressive Sensing](https://mlsanity.com/paper/2310.00420/) (2023)

## Abstract (arXiv)

Sparse Bayesian learning (SBL) is a powerful framework for tackling the sparse coding problem while also providing uncertainty quantification. The most popular inference algorithms for SBL exhibit prohibitively large computational costs for high-dimensional problems due to the need to maintain a large covariance matrix. To resolve this issue, we introduce a new method for accelerating SBL inference -- named covariance-free expectation maximization (CoFEM) -- that avoids explicit computation of the covariance matrix. CoFEM solves multiple linear systems to obtain unbiased estimates of the posterior statistics needed by SBL. This is accomplished by exploiting innovations from numerical linear algebra such as preconditioned conjugate gradient and a little-known diagonal estimation rule. For a large class of compressed sensing matrices, we provide theoretical justifications for why our method scales well in high-dimensional settings. Through simulations, we show that CoFEM can be up to thousands of times faster than existing baselines without sacrificing coding accuracy. Through applications to calcium imaging deconvolution and multi-contrast MRI reconstruction, we show that CoFEM enables SBL to tractably tackle high-dimensional sparse coding problems of practical interest.