mlsanityData to 7 Oct 2026

Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition

arXiv 2106.05992 ↗First posted 10 Jun 2021cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We propose the harmonic kernel decomposition (HKD), which uses Fourier series to decompose a kernel as a sum of orthogonal kernels. Our variational approximation exploits this orthogonality to enable a large number of inducing points at a low computational cost. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections, and it significantly outperforms standard variational methods in scalability and accuracy. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 10 Jun 2021.

Builds on

25 of 46 references.

References · 25

  1. 2013Gaussian Processes for Big Datacited by 377
  2. 2014Scalable Variational Gaussian Process Classificationcited by 226
  3. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 175
  4. 2017Doubly Stochastic Variational Inference for Deep Gaussian Processescited by 143
  5. 2013Gaussian Process Kernels for Pattern Discovery and Extrapolationcited by 131
  6. 2019Exact Gaussian Processes on a Million Data Pointscited by 118
  7. 2016Orthogonal Random Featurescited by 89
  8. 2014Hilbert Space Methods for Reduced-Rank Gaussian Process Regressioncited by 88
  9. 2016Variational Fourier features for Gaussian processescited by 85
  10. 2020Mat'ern Gaussian processes on Riemannian manifoldscited by 82
  11. 2019Rates of Convergence for Sparse Variational Gaussian Process Regressioncited by 62
  12. 2018Learning Invariances using the Marginal Likelihoodcited by 51
13 more
  1. 2017Convolutional Gaussian Processescited by 44
  2. 2020Sparse Gaussian Processes with Spherical Harmonic Featurescited by 43
  3. 2017Spectral Mixture Kernels for Multi-Output Gaussian Processescited by 29
  4. 2017Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decompositioncited by 28
  5. 2015Thoughts on Massively Scalable Gaussian Processescited by 26
  6. 2017Gaussian Quadrature for Kernel Featurescited by 21
  7. 2017Variational Inference for Gaussian Process Models with Linear Complexitycited by 21
  8. 2018Deep convolutional Gaussian processescited by 21
  9. 2018Orthogonally Decoupled Variational Gaussian Processescited by 19
  10. 2015Generalized Spectral Kernelscited by 17
  11. 2019Function-Space Distributions over Kernelscited by 17
  12. 2018Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)cited by 9
  13. 2020Variational Orthogonal Featurescited by 9

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 0; 2023: 3; 2024: 1; 2025: 2.

8 papers.

  1. 2023Calibrating Transformers via Sparse Gaussian Processescited by 21
  2. 2024Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inferencecited by 16
  3. 2021When are Iterative Gaussian Processes Reliably Accurate?cited by 12
  4. 2025Tighter sparse variational Gaussian processescited by 3
  5. 2021Nonnegative spatial factorizationcited by 2
  6. 2023Sparse Gaussian Processes with Spherical Harmonic Features Revisitedcited by 2
  7. 2025New Bounds for Sparse Variational Gaussian Processescited by 2
  8. 2023Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processescited by 1