# Variance Reduction for Matrix Computations with Applications to Gaussian Processes

arXiv: https://arxiv.org/abs/2106.14565 · HTML: https://mlsanity.com/paper/2106.14565/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Hutch++: Optimal Stochastic Trace Estimation](https://mlsanity.com/paper/2010.09649/) (2020)
- [Equilibrated adaptive learning rates for non-convex optimization](https://mlsanity.com/paper/1502.04390/) (2015)
- [Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization](https://mlsanity.com/paper/2006.11267/) (2020)
- [Estimating the Spectral Density of Large Implicit Matrices](https://mlsanity.com/paper/1802.03451/) (2018)
- [Scalable Gaussian Process Computations Using Hierarchical Matrices](https://mlsanity.com/paper/1808.03215/) (2018)
- [Improved stochastic trace estimation using mutually unbiased bases](https://mlsanity.com/paper/1608.00117/) (2016)

## Abstract (arXiv)

In addition to recent developments in computing speed and memory, methodological advances have contributed to significant gains in the performance of stochastic simulation. In this paper, we focus on variance reduction for matrix computations via matrix factorization. We provide insights into existing variance reduction methods for estimating the entries of large matrices. Popular methods do not exploit the reduction in variance that is possible when the matrix is factorized. We show how computing the square root factorization of the matrix can achieve in some important cases arbitrarily better stochastic performance. In addition, we propose a factorized estimator for the trace of a product of matrices and numerically demonstrate that the estimator can be up to 1,000 times more efficient on certain problems of estimating the log-likelihood of a Gaussian process. Additionally, we provide a new estimator of the log-determinant of a positive semi-definite matrix where the log-determinant is treated as a normalizing constant of a probability density.