# Stein Random Feature Regression

arXiv: https://arxiv.org/abs/2406.00438 · HTML: https://mlsanity.com/paper/2406.00438/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond](https://mlsanity.com/paper/2004.11154/) (2020)
- [Orthogonal Random Features](https://mlsanity.com/paper/1610.09072/) (2016)
- [Variational Fourier features for Gaussian processes](https://mlsanity.com/paper/1611.06740/) (2016)
- [Non-Stationary Spectral Kernels](https://mlsanity.com/paper/1705.08736/) (2017)
- [Random Feature Expansions for Deep Gaussian Processes](https://mlsanity.com/paper/1610.04386/) (2016)
- [Stein Variational Gradient Descent With Matrix-Valued Kernels](https://mlsanity.com/paper/1910.12794/) (2019)
- [On Stein Variational Neural Network Ensembles](https://mlsanity.com/paper/2106.10760/) (2021)
- [Learning to Learn Kernels with Variational Random Features](https://mlsanity.com/paper/2006.06707/) (2020)
- [Function-Space Distributions over Kernels](https://mlsanity.com/paper/1910.13565/) (2019)
- [Spatial Mapping with Gaussian Processes and Nonstationary Fourier Features](https://mlsanity.com/paper/1711.05615/) (2017)
- [Harmonizable mixture kernels with variational Fourier features](https://mlsanity.com/paper/1810.04416/) (2018)
- [Deep Kernel Learning via Random Fourier Features](https://mlsanity.com/paper/1910.02660/) (2019)
- [Deep Kernels with Probabilistic Embeddings for Small-Data Learning](https://mlsanity.com/paper/1910.05858/) (2019)
- [Data-driven Random Fourier Features using Stein Effect](https://mlsanity.com/paper/1705.08525/) (2017)
- [Marginalising over Stationary Kernels with Bayesian Quadrature](https://mlsanity.com/paper/2106.07452/) (2021)
- [Integrated Variational Fourier Features for Fast Spatial Modelling with Gaussian Processes](https://mlsanity.com/paper/2308.14142/) (2023)

## Abstract (arXiv)

In large-scale regression problems, random Fourier features (RFFs) have significantly enhanced the computational scalability and flexibility of Gaussian processes (GPs) by defining kernels through their spectral density, from which a finite set of Monte Carlo samples can be used to form an approximate low-rank GP. However, the efficacy of RFFs in kernel approximation and Bayesian kernel learning depends on the ability to tractably sample the kernel spectral measure and the quality of the generated samples. We introduce Stein random features (SRF), leveraging Stein variational gradient descent, which can be used to both generate high-quality RFF samples of known spectral densities as well as flexibly and efficiently approximate traditionally non-analytical spectral measure posteriors. SRFs require only the evaluation of log-probability gradients to perform both kernel approximation and Bayesian kernel learning that results in superior performance over traditional approaches. We empirically validate the effectiveness of SRFs by comparing them to baselines on kernel approximation and well-known GP regression problems.