mlsanityData to 7 Oct 2026

Faster Kernel Interpolation for Gaussian Processes

arXiv 2101.11751 ↗First posted 28 Jan 2021cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

A key challenge in scaling Gaussian Process (GP) regression to massive datasets is that exact inference requires computation with a dense n x n kernel matrix, where n is the number of data points. Significant work focuses on approximating the kernel matrix via interpolation using a smaller set of m inducing points. Structured kernel interpolation (SKI) is among the most scalable methods: by placing inducing points on a dense grid and using structured matrix algebra, SKI achieves per-iteration time of O(n + m log m) for approximate inference. This linear scaling in n enables inference for very large data sets; however the cost is per-iteration, which remains a limitation for extremely large n. We show that the SKI per-iteration time can be reduced to O(m log m) after a single O(n) time precomputation step by reframing SKI as solving a natural Bayesian linear regression problem with a fixed set of m compact basis functions. With per-iteration complexity independent of the dataset size n for a fixed grid, our method scales to truly massive data sets. We demonstrate speedups in practice for a wide range of m and n and apply the method to GP inference on a three-dimensional weather radar dataset with over 100 million points.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 16 of its 37 references that are in the map. First posted on arXiv 28 Jan 2021.

Builds on

16 of 37 references.

References · 16

  1. 2017Deep Neural Networks as Gaussian Processescited by 617
  2. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  3. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  4. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 175
  5. 2018Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processescited by 164
  6. 2018Deep Convolutional Networks as shallow Gaussian Processescited by 137
  7. 2013Gaussian Process Kernels for Pattern Discovery and Extrapolationcited by 131
  8. 2020Kernel methods through the roof: handling billions of points efficientlycited by 88
  9. 2014Fastfood: Approximate Kernel Expansions in Loglinear Timecited by 72
  10. 2019A Bayesian Perspective on the Deep Image Priorcited by 52
  11. 2018Constant-Time Predictive Distributions for Gaussian Processescited by 48
  12. 2017A Case Study Competition Among Methods for Analyzing Large Spatial Datacited by 44
4 more
  1. 2017Scalable Log Determinants for Gaussian Process Kernel Learningcited by 34
  2. 2018Product Kernel Interpolation for Scalable Gaussian Processescited by 31
  3. 2015Thoughts on Massively Scalable Gaussian Processescited by 26
  4. 2014Bayesian and Maximum Likelihood Estimation for Gaussian Processes on an Incomplete Latticecited by 7

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 3.

5 papers.

  1. 2024Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networkscited by 18
  2. 2023Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regressioncited by 5
  3. 2022Spatially scalable recursive estimation of Gaussian process terrain maps using local basis functionscited by 2
  4. 2024Tensor Network-Constrained Kernel Machines as Gaussian Processescited by 2
  5. 2024Exploiting Hankel-Toeplitz Structures for Fast Computation of Kernel Precision Matricescited by 0