# Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era

arXiv: https://arxiv.org/abs/1902.10078 · HTML: https://mlsanity.com/paper/1902.10078/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2019.

## Built on by (most cited first, 11 of 11)

- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Sparse Algorithms for Markovian Gaussian Processes](https://mlsanity.com/paper/2103.10710/) (2021)
- [Correlated Product of Experts for Sparse Gaussian Process Regression](https://mlsanity.com/paper/2112.09519/) (2021)
- [Entropy-based adaptive Hamiltonian Monte Carlo](https://mlsanity.com/paper/2110.14625/) (2021)
- [Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees](https://mlsanity.com/paper/2210.07893/) (2022)
- [Actually Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2304.05091/) (2023)
- [Non-stationary Gaussian process discriminant analysis with variable selection for high-dimensional functional data](https://mlsanity.com/paper/2109.14171/) (2021)
- [Scalable Deep Gaussian Markov Random Fields for General Graphs](https://mlsanity.com/paper/2206.05032/) (2022)
- [Contraction rates for conjugate gradient and Lanczos approximate posteriors in Gaussian process regression](https://mlsanity.com/paper/2406.12678/) (2024)
- [On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach](https://mlsanity.com/paper/2103.03321/) (2021)
- [A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression](https://mlsanity.com/paper/2608.11917/) (2026)

## Abstract (arXiv)

Banded matrices can be used as precision matrices in several models including linear state-space models, some Gaussian processes, and Gaussian Markov random fields. The aim of the paper is to make modern inference methods (such as variational inference or gradient-based sampling) available for Gaussian models with banded precision. We show that this can efficiently be achieved by equipping an automatic differentiation framework, such as TensorFlow or PyTorch, with some linear algebra operators dedicated to banded matrices. This paper studies the algorithmic aspects of the required operators, details their reverse-mode derivatives, and show that their complexity is linear in the number of observations.