# A Joint introduction to Gaussian Processes and Relevance Vector Machines with Connections to Kalman filtering and other Kernel Smoothers

arXiv: https://arxiv.org/abs/2009.09217 · HTML: https://mlsanity.com/paper/2009.09217/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2020.

## Built on by (most cited first, 10 of 10)

- [A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABC](https://mlsanity.com/paper/2108.00490/) (2021)
- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)
- [Edge Tracing using Gaussian Process Regression](https://mlsanity.com/paper/2111.03605/) (2021)
- [On the safe use of prior densities for Bayesian model selection](https://mlsanity.com/paper/2206.05210/) (2022)
- [MCMC-driven importance samplers](https://mlsanity.com/paper/2105.02579/) (2021)
- [Spectral information criterion for automatic elbow detection](https://mlsanity.com/paper/2308.09108/) (2023)
- [Hyperbolic Secant representation of the logistic function: Application to probabilistic Multiple Instance Learning for CT intracranial hemorrhage detection](https://mlsanity.com/paper/2403.14829/) (2024)
- [Data-driven informative priors for Bayesian inference with quasi-periodic data](https://mlsanity.com/paper/2511.22296/) (2025)
- [A note on the area under the likelihood and the fake evidence for model selection](https://mlsanity.com/paper/2602.22965/) (2026)
- [An automatic counting algorithm for the quantification and uncertainty analysis of the number of microglial cells trainable in small and heterogeneous datasets](https://mlsanity.com/paper/2602.22974/) (2026)

## Abstract (arXiv)

The expressive power of Bayesian kernel-based methods has led them to become an important tool across many different facets of artificial intelligence, and useful to a plethora of modern application domains, providing both power and interpretability via uncertainty analysis. This article introduces and discusses two methods which straddle the areas of probabilistic Bayesian schemes and kernel methods for regression: Gaussian Processes and Relevance Vector Machines. Our focus is on developing a common framework with which to view these methods, via intermediate methods a probabilistic version of the well-known kernel ridge regression, and drawing connections among them, via dual formulations, and discussion of their application in the context of major tasks: regression, smoothing, interpolation, and filtering. Overall, we provide understanding of the mathematical concepts behind these models, and we summarize and discuss in depth different interpretations and highlight the relationship to other methods, such as linear kernel smoothers, Kalman filtering and Fourier approximations. Throughout, we provide numerous figures to promote understanding, and we make numerous recommendations to practitioners. Benefits and drawbacks of the different techniques are highlighted. To our knowledge, this is the most in-depth study of its kind to date focused on these two methods, and will be relevant to theoretical understanding and practitioners throughout the domains of data-science, signal processing, machine learning, and artificial intelligence in general.