# Shallow and Deep Nonparametric Convolutions for Gaussian Processes

arXiv: https://arxiv.org/abs/2206.08972 · HTML: https://mlsanity.com/paper/2206.08972/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Deep Gaussian Processes for Regression using Approximate Expectation Propagation](https://mlsanity.com/paper/1602.04133/) (2016)
- [Spectral Mixture Kernels for Multi-Output Gaussian Processes](https://mlsanity.com/paper/1709.01298/) (2017)
- [Deep convolutional Gaussian processes](https://mlsanity.com/paper/1810.03052/) (2018)
- [Function-Space Distributions over Kernels](https://mlsanity.com/paper/1910.13565/) (2019)
- [Deep learning with differential Gaussian process flows](https://mlsanity.com/paper/1810.04066/) (2018)
- [Harmonizable mixture kernels with variational Fourier features](https://mlsanity.com/paper/1810.04416/) (2018)
- [Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features](https://mlsanity.com/paper/2106.05960/) (2021)
- [Nonstationary multi-output Gaussian processes via harmonizable spectral mixtures](https://mlsanity.com/paper/2202.09233/) (2022)
- [Scalable L'evy Process Priors for Spectral Kernel Learning](https://mlsanity.com/paper/1802.00530/) (2018)
- [Learning Nonparametric Volterra Kernels with Gaussian Processes](https://mlsanity.com/paper/2106.05582/) (2021)
- [Learning spectrograms with convolutional spectral kernels](https://mlsanity.com/paper/1905.09917/) (2019)
- [Modelling Non-Smooth Signals with Complex Spectral Structure](https://mlsanity.com/paper/2203.06997/) (2022)

## Built on by (most cited first, 1 of 1)

- [Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learning](https://mlsanity.com/paper/2311.14828/) (2023)

## Abstract (arXiv)

A key challenge in the practical application of Gaussian processes (GPs) is selecting a proper covariance function. The moving average, or process convolutions, construction of GPs allows some additional flexibility, but still requires choosing a proper smoothing kernel, which is non-trivial. Previous approaches have built covariance functions by using GP priors over the smoothing kernel, and by extension the covariance, as a way to bypass the need to specify it in advance. However, such models have been limited in several ways: they are restricted to single dimensional inputs, e.g. time; they only allow modelling of single outputs and they do not scale to large datasets since inference is not straightforward. In this paper, we introduce a nonparametric process convolution formulation for GPs that alleviates these weaknesses by using a functional sampling approach based on Matheron's rule to perform fast sampling using interdomain inducing variables. Furthermore, we propose a composition of these nonparametric convolutions that serves as an alternative to classic deep GP models, and allows the covariance functions of the intermediate layers to be inferred from the data. We test the performance of our model on benchmarks for single output GPs, multiple output GPs and deep GPs and find that our approach can provide improvements over standard GP models, particularly for larger datasets.