Accurate and Scalable Stochastic Gaussian Process Regression via Learnable Coreset-based Variational Inference

We introduce a novel stochastic variational inference method for Gaussian process (GP) regression, by deriving a posterior over a learnable set of coresets: i.e., over pseudo-input/output, weighted pairs. Unlike former free-form variational families for stochastic inference, our coreset-based variational GP (CVGP) is defined in terms of the GP prior and the (weighted) data likelihood. This formulation naturally incorporates inductive biases of the prior, and ensures its kernel and likelihood dependencies are shared with the posterior. We derive a variational lower-bound on the log-marginal likelihood by marginalizing over the latent GP coreset variables, and show that CVGP's lower-bound is amenable to stochastic optimization. CVGP reduces the dimensionality of the variational parameter search space to linear O(M) complexity, while ensuring numerical stability at O(M3) time complexity and O(M2) space complexity.