mlsanityData to 7 Oct 2026

On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes

arXiv 1504.07027 ↗First posted 27 Apr 2015stat.MLcited by 56

Not on the map. Why?

Abstract (arXiv)

The variational framework for learning inducing variables (Titsias, 2009a) has had a large impact on the Gaussian process literature. The framework may be interpreted as minimizing a rigorously defined Kullback-Leibler divergence between the approximating and posterior processes. To our knowledge this connection has thus far gone unremarked in the literature. In this paper we give a substantial generalization of the literature on this topic. We give a new proof of the result for infinite index sets which allows inducing points that are not data points and likelihoods that depend on all function values. We then discuss augmented index sets and show that, contrary to previous works, marginal consistency of augmentation is not enough to guarantee consistency of variational inference with the original model. We then characterize an extra condition where such a guarantee is obtainable. Finally we show how our framework sheds light on interdomain sparse approximations and sparse approximations for Cox processes.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Apr 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 11; 2023: 17; 2024: 6; 2025: 7; 2026: 1.

56 papers.

Sort:

By area:

  1. 2023CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or Notcited by 79
  2. 2022Diffusion Generative Models in Infinite Dimensionscited by 49
  3. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
  4. 2021The Gaussian Neural Processcited by 34
  5. 2023Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisationcited by 32
  6. 2023Continual Learning via Sequential Function-Space Variational Inferencecited by 28
  7. 2023Sampling from Gaussian Process Posteriors using Stochastic Gradient Descentcited by 27
  8. 2021Deep Neural Networks as Point Estimates for Deep Gaussian Processescited by 26
  9. 2021Meta-Learning Reliable Priors in the Function Spacecited by 23
  10. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  11. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  12. 2022Generalized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learningcited by 20
38 more
  1. 2022Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learningcited by 18
  2. 2021Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processescited by 17
  3. 2023Exploiting Unlabelled Photos for Stronger Fine-Grained SBIRcited by 14
  4. 2021Variational Inference for Continuous-Time Switching Dynamical Systemscited by 9
  5. 2022Wide Bayesian neural networks have a simple weight posterior: theory and accelerated samplingcited by 9
  6. 2024Approximation-Aware Bayesian Optimizationcited by 8
  7. 2021Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processescited by 7
  8. 2022Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Treescited by 5
  9. 2023FaIRGP: A Bayesian Energy Balance Model for Surface Temperatures Emulationcited by 5
  10. 2022Sparse Gaussian Process Hyperparameters: Optimize or Integrate?cited by 4
  11. 2023On permutation symmetries in Bayesian neural network posteriors: a variational perspectivecited by 4
  12. 2025NeuralSurv: Deep Survival Analysis with Bayesian Uncertainty Quantificationcited by 4
  13. 2021Estimation of Riemannian distances between covariance operators and Gaussian processescited by 3
  14. 2021Variational Gaussian Processes: A Functional Analysis Viewcited by 3
  15. 2023Integrated Variational Fourier Features for Fast Spatial Modelling with Gaussian Processescited by 3
  16. 2025Tighter sparse variational Gaussian processescited by 3
  17. 2021Active Learning in Gaussian Process State Space Modelcited by 2
  18. 2022Modelling Non-Smooth Signals with Complex Spectral Structurecited by 2
  19. 2022Causal Modeling of Policy Interventions From Sequences of Treatments and Outcomescited by 2
  20. 2023A Unifying Variational Framework for Gaussian Process Motion Planningcited by 2
  21. 2023Entropic Matching for Expectation Propagation of Markov Jump Processescited by 2
  22. 2023Function Space Bayesian Pseudocoreset for Bayesian Neural Networkscited by 2
  23. 2023Variational Gaussian Processes For Linear Inverse Problemscited by 2
  24. 2024Recommendations for Baselines and Benchmarking Approximate Gaussian Processescited by 2
  25. 2024Functional Stochastic Gradient MCMC for Bayesian Neural Networkscited by 2
  26. 2025New Bounds for Sparse Variational Gaussian Processescited by 2
  27. 2021On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approachcited by 1
  28. 2021Towards Fully Automated Segmentation of Rat Cardiac MRI by Leveraging Deep Learning Frameworkscited by 1
  29. 2021Conditional Deep Gaussian Processes: empirical Bayes hyperdata learningcited by 1
  30. 2021Modular Gaussian Processes for Transfer Learningcited by 1
  31. 2022Variational Inference for Model-Free and Model-Based Reinforcement Learningcited by 1
  32. 2023Efficient Sensor Placement from Regression with Sparse Gaussian Processes in Continuous and Discrete Spacescited by 1
  33. 2023Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion priorcited by 1
  34. 2024One-Shot Federated Learning with Bayesian Pseudocoresetscited by 1
  35. 2024Adjusting Model Size in Continual Gaussian Processes: How Big is Big Enough?cited by 1
  36. 2025Robust and Computation-Aware Gaussian Processescited by 1
  37. 2022AODisaggregation: toward global aerosol vertical profilescited by 0
  38. 2023On the Calibration and Uncertainty with P'olya-Gamma Augmentation for Dialog Retrieval Modelscited by 0

50 most cited of 56 All 56 (JSON)