# Hilbert Space Methods for Reduced-Rank Gaussian Process Regression

arXiv: https://arxiv.org/abs/1401.5508 · HTML: https://mlsanity.com/paper/1401.5508/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2014.

## Built on by (most cited first, 30 of 88)

- [Deep Neural Networks as Point Estimates for Deep Gaussian Processes](https://mlsanity.com/paper/2105.04504/) (2021)
- [Pathfinder: Parallel quasi-Newton variational inference](https://mlsanity.com/paper/2108.03782/) (2021)
- [Periodic Activation Functions Induce Stationarity](https://mlsanity.com/paper/2110.13572/) (2021)
- [High-Dimensional Gaussian Process Inference with Derivatives](https://mlsanity.com/paper/2102.07542/) (2021)
- [Large-Scale Learning with Fourier Features and Tensor Decompositions](https://mlsanity.com/paper/2109.01545/) (2021)
- [Improved Regret Bounds for Gaussian Process Upper Confidence Bound in Bayesian Optimization](https://mlsanity.com/paper/2506.01393/) (2025)
- [When are Iterative Gaussian Processes Reliably Accurate?](https://mlsanity.com/paper/2112.15246/) (2021)
- [The GeometricKernels Package: Heat and Mat'ern Kernels for Geometric Learning on Manifolds, Meshes, and Graphs](https://mlsanity.com/paper/2407.08086/) (2024)
- [Sparse Algorithms for Markovian Gaussian Processes](https://mlsanity.com/paper/2103.10710/) (2021)
- [Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition](https://mlsanity.com/paper/2106.05992/) (2021)
- [Physics-informed machine learning for Structural Health Monitoring](https://mlsanity.com/paper/2206.15303/) (2022)
- [Constraining Gaussian processes for physics-informed acoustic emission mapping](https://mlsanity.com/paper/2206.01495/) (2022)
- [Discussing the Spectrum of Physics-Enhanced Machine Learning; a Survey on Structural Mechanics Applications](https://mlsanity.com/paper/2310.20425/) (2023)
- [Quantized Fourier and Polynomial Features for more Expressive Tensor Network Models](https://mlsanity.com/paper/2309.05436/) (2023)
- [Implicit Gaussian process representation of vector fields over arbitrary latent manifolds](https://mlsanity.com/paper/2309.16746/) (2023)
- [Efficient Exploration in Binary and Preferential Bayesian Optimization](https://mlsanity.com/paper/2110.09361/) (2021)
- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)
- [Dynamic Online Ensembles of Basis Expansions](https://mlsanity.com/paper/2405.01365/) (2024)
- [Optimizing Posterior Samples for Bayesian Optimization via Rootfinding](https://mlsanity.com/paper/2410.22322/) (2024)
- [Gaussian Processes on Hypergraphs](https://mlsanity.com/paper/2106.01982/) (2021)
- [Quantum-Assisted Hilbert-Space Gaussian Process Regression](https://mlsanity.com/paper/2402.00544/) (2024)
- [Efficient Deconvolution in Populational Inverse Problems](https://mlsanity.com/paper/2505.19841/) (2025)
- [Conditioning Gaussian Processes on Almost Anything](https://mlsanity.com/paper/2605.21041/) (2026)
- [Low-rank statistical finite elements for scalable model-data synthesis](https://mlsanity.com/paper/2109.04757/) (2021)
- [Orthonormal Expansions for Translation-Invariant Kernels](https://mlsanity.com/paper/2206.08648/) (2022)
- [Learning-Based Optimal Control with Performance Guarantees for Unknown Systems with Latent States](https://mlsanity.com/paper/2303.17963/) (2023)
- [Integrated Variational Fourier Features for Fast Spatial Modelling with Gaussian Processes](https://mlsanity.com/paper/2308.14142/) (2023)
- [Out of Distribution Detection via Domain-Informed Gaussian Process State Space Models](https://mlsanity.com/paper/2309.06655/) (2023)
- [Latent mixed-effect models for high-dimensional longitudinal data](https://mlsanity.com/paper/2409.11008/) (2024)
- [Provable Quantum Algorithm Advantage for Gaussian Process Quadrature](https://mlsanity.com/paper/2502.14467/) (2025)

## Abstract (arXiv)

This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction expansion of the Laplace operator in a compact subset of ℝd. On this approximate eigenbasis the eigenvalues of the covariance function can be expressed as simple functions of the spectral density of the Gaussian process, which allows the GP inference to be solved under a computational cost scaling as O(nm2) (initial) and O(m3) (hyperparameter learning) with m basis functions and n data points. Furthermore, the basis functions are independent of the parameters of the covariance function, which allows for very fast hyperparameter learning. The approach also allows for rigorous error analysis with Hilbert space theory, and we show that the approximation becomes exact when the size of the compact subset and the number of eigenfunctions go to infinity. We also show that the convergence rate of the truncation error is independent of the input dimensionality provided that the differentiability order of the covariance function is increases appropriately, and for the squared exponential covariance function it is always bounded by ~1/m regardless of the input dimensionality. The expansion generalizes to Hilbert spaces with an inner product which is defined as an integral over a specified input density. The method is compared to previously proposed methods theoretically and through empirical tests with simulated and real data.