mlsanityData to 7 Oct 2026

Dual Parameterization of Sparse Variational Gaussian Processes

arXiv 2111.03412 ↗First posted 5 Nov 2021cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by using a dual parameterization where each data example is assigned dual parameters, similarly to site parameters used in expectation propagation. Our dual parameterization speeds-up inference using natural gradient descent, and provides a tighter evidence lower bound for hyperparameter learning. The approach has the same memory cost as the current SVGP methods, but it is faster and more accurate.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 16 of its 40 references that are in the map. First posted on arXiv 5 Nov 2021.

Builds on

16 of 40 references.

References · 16

  1. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  2. 2013Gaussian Processes for Big Datacited by 377
  3. 2014Scalable Variational Gaussian Process Classificationcited by 226
  4. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  5. 2016GPflow: A Gaussian process library using TensorFlowcited by 139
  6. 2018Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adamcited by 131
  7. 2017Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Modelscited by 65
  8. 2020A Framework for Interdomain and Multioutput Gaussian Processescited by 47
  9. 2018Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Modelscited by 38
  10. 2018Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Modelscited by 37
  11. 2015Stochastic Expectation Propagationcited by 22
  12. 2017Variational Inference for Gaussian Process Models with Linear Complexitycited by 21
4 more
  1. 2020Fast Variational Learning in State-Space Gaussian Process Modelscited by 20
  2. 2015Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functionscited by 19
  3. 2018Orthogonally Decoupled Variational Gaussian Processescited by 19
  4. 2021Sparse Algorithms for Markovian Gaussian Processescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 3; 2025: 5; 2026: 3.

19 papers.

Sort:

  1. 2022Sampling-based inference for large linear models, with application to linearised Laplacecited by 18
  2. 2023Memory-Based Dual Gaussian Processes for Sequential Learningcited by 11
  3. 2024Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systemscited by 11
  4. 2023Variational Gaussian Process Diffusion Processescited by 10
  5. 2022Fantasizing with Dual GPs in Bayesian Optimization and Active Learningcited by 7
  6. 2024Function-space Parameterization of Neural Networks for Sequential Learningcited by 7
  7. 2025In-Context Learning of Stochastic Differential Equations with Foundation Inference Modelscited by 7
  8. 2025Knowledge Adaptation as Posterior Correctioncited by 6
  9. 2022Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Treescited by 5
  10. 2025SING: SDE Inference via Natural Gradientscited by 4
  11. 2023Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Modelscited by 3
  12. 2025Federated ADMM from Bayesian Dualitycited by 3
7 more
  1. 2025Optimization Guarantees for Square-Root Natural-Gradient Variational Inferencecited by 3
  2. 2026Foundation Inference Models for Ordinary Differential Equationscited by 3
  3. 2023Sparse Function-space Representation of Neural Networkscited by 1
  4. 2024One-Shot Federated Learning with Bayesian Pseudocoresetscited by 1
  5. 2022Towards Improved Learning in Gaussian Processes: The Best of Two Worldscited by 0
  6. 2026Inverse-Free Sparse Variational Gaussian Processescited by 0
  7. 2026A Generalization of Amari's Bayesian Dualitycited by 0