# Variational Relevance Vector Machines

arXiv: https://arxiv.org/abs/1301.3838 · HTML: https://mlsanity.com/paper/1301.3838/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2013.

## Built on by (most cited first, 21 of 21)

- [Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2102.00397/) (2021)
- [Semi-Implicit Variational Inference via Score Matching](https://mlsanity.com/paper/2308.10014/) (2023)
- [Canonical normalizing flows for manifold learning](https://mlsanity.com/paper/2310.12743/) (2023)
- [Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration](https://mlsanity.com/paper/2310.17153/) (2023)
- [Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertainty](https://mlsanity.com/paper/2210.08095/) (2022)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [Weak neural variational inference for solving Bayesian inverse problems without forward models: applications in elastography](https://mlsanity.com/paper/2407.20697/) (2024)
- [Adaptive sparseness for correntropy-based robust regression via automatic relevance determination](https://mlsanity.com/paper/2302.00082/) (2023)
- [Sparse online variational Bayesian regression](https://mlsanity.com/paper/2102.12261/) (2021)
- [Covariance-Free Sparse Bayesian Learning](https://mlsanity.com/paper/2105.10439/) (2021)
- [High-Dimensional Sparse Bayesian Learning without Covariance Matrices](https://mlsanity.com/paper/2202.12808/) (2022)
- [Variational Graph Autoencoder for Heterogeneous Information Networks with Missing and Inaccurate Attributes](https://mlsanity.com/paper/2311.07929/) (2023)
- [Sparse Bayesian Correntropy Learning for Robust Muscle Activity Reconstruction from Noisy Brain Recordings](https://mlsanity.com/paper/2404.15309/) (2024)
- [Semi-Implicit Functional Gradient Flow for Efficient Sampling](https://mlsanity.com/paper/2410.17935/) (2024)
- [A Kernel Approach for Semi-implicit Variational Inference](https://mlsanity.com/paper/2601.12023/) (2026)
- [Analyzing Relevance Vector Machines using a single penalty approach](https://mlsanity.com/paper/2107.02085/) (2021)
- [Posterior Consistency for Bayesian Relevance Vector Machines](https://mlsanity.com/paper/2202.05422/) (2022)
- [RENs: Relevance Encoding Networks](https://mlsanity.com/paper/2205.13061/) (2022)
- [The Relevance Feature and Vector Machine for health applications](https://mlsanity.com/paper/2402.07079/) (2024)
- [The Illusion of Fit: Spatially Resolved Assessment of Constitutive Model Validity in Elastography and Physics-Based Inverse Problems](https://mlsanity.com/paper/2502.07415/) (2025)
- [From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs](https://mlsanity.com/paper/2609.28942/) (2026)

## Abstract (arXiv)

The Support Vector Machine (SVM) of Vapnik (1998) has become widely established as one of the leading approaches to pattern recognition and machine learning. It expresses predictions in terms of a linear combination of kernel functions centred on a subset of the training data, known as support vectors. Despite its widespread success, the SVM suffers from some important limitations, one of the most significant being that it makes point predictions rather than generating predictive distributions. Recently Tipping (1999) has formulated the Relevance Vector Machine (RVM), a probabilistic model whose functional form is equivalent to the SVM. It achieves comparable recognition accuracy to the SVM, yet provides a full predictive distribution, and also requires substantially fewer kernel functions. The original treatment of the RVM relied on the use of type II maximum likelihood (the `evidence framework') to provide point estimates of the hyperparameters which govern model sparsity. In this paper we show how the RVM can be formulated and solved within a completely Bayesian paradigm through the use of variational inference, thereby giving a posterior distribution over both parameters and hyperparameters. We demonstrate the practicality and performance of the variational RVM using both synthetic and real world examples.