# Function-Space Distributions over Kernels

arXiv: https://arxiv.org/abs/1910.13565 · HTML: https://mlsanity.com/paper/1910.13565/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2019.

## Built on by (most cited first, 17 of 17)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning](https://mlsanity.com/paper/2304.05366/) (2023)
- [Deep Learning is Not So Mysterious or Different](https://mlsanity.com/paper/2503.02113/) (2025)
- [MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting](https://mlsanity.com/paper/2306.08777/) (2023)
- [Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition](https://mlsanity.com/paper/2106.05992/) (2021)
- [Structural Kernel Search via Bayesian Optimization and Symbolical Optimal Transport](https://mlsanity.com/paper/2210.11836/) (2022)
- [Characteristic kernels on Hilbert spaces, Banach spaces, and on sets of measures](https://mlsanity.com/paper/2206.07588/) (2022)
- [Learning Nonparametric Volterra Kernels with Gaussian Processes](https://mlsanity.com/paper/2106.05582/) (2021)
- [Marginalising over Stationary Kernels with Bayesian Quadrature](https://mlsanity.com/paper/2106.07452/) (2021)
- [Modelling Non-Smooth Signals with Complex Spectral Structure](https://mlsanity.com/paper/2203.06997/) (2022)
- [On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel](https://mlsanity.com/paper/2203.07411/) (2022)
- [Shallow and Deep Nonparametric Convolutions for Gaussian Processes](https://mlsanity.com/paper/2206.08972/) (2022)
- [Volatility Based Kernels and Moving Average Means for Accurate Forecasting with Gaussian Processes](https://mlsanity.com/paper/2207.06544/) (2022)
- [Spectrum Gaussian Processes Based On Tunable Basis Functions](https://mlsanity.com/paper/2107.06473/) (2021)
- [Physics Inspired Approaches To Understanding Gaussian Processes](https://mlsanity.com/paper/2305.10748/) (2023)
- [Stein Random Feature Regression](https://mlsanity.com/paper/2406.00438/) (2024)
- [Regular Fourier Features for Nonstationary Gaussian Processes](https://mlsanity.com/paper/2602.23006/) (2026)

## Abstract (arXiv)

Gaussian processes are flexible function approximators, with inductive biases controlled by a covariance kernel. Learning the kernel is the key to representation learning and strong predictive performance. In this paper, we develop functional kernel learning (FKL) to directly infer functional posteriors over kernels. In particular, we place a transformed Gaussian process over a spectral density, to induce a non-parametric distribution over kernel functions. The resulting approach enables learning of rich representations, with support for any stationary kernel, uncertainty over the values of the kernel, and an interpretable specification of a prior directly over kernels, without requiring sophisticated initialization or manual intervention. We perform inference through elliptical slice sampling, which is especially well suited to marginalizing posteriors with the strongly correlated priors typical to function space modelling. We develop our approach for non-uniform, large-scale, multi-task, and multidimensional data, and show promising performance in a wide range of settings, including interpolation, extrapolation, and kernel recovery experiments.