# Revisiting Active Sets for Gaussian Process Decoders

arXiv: https://arxiv.org/abs/2209.04636 · HTML: https://mlsanity.com/paper/2209.04636/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 10 Sep 2022.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [PAC-Bayesian Theory Meets Bayesian Inference](https://mlsanity.com/paper/1605.08636/) (2016)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [Convolutional Gaussian Processes](https://mlsanity.com/paper/1709.01894/) (2017)
- [On the marginal likelihood and cross-validation](https://mlsanity.com/paper/1905.08737/) (2019)
- [Amortized Inference Regularization](https://mlsanity.com/paper/1805.08913/) (2018)
- [Latent Gaussian Processes for Distribution Estimation of Multivariate Categorical Data](https://mlsanity.com/paper/1503.02182/) (2015)

## Built on by (most cited first, 5 of 5)

- [On Masked Pre-training and the Marginal Likelihood](https://mlsanity.com/paper/2306.00520/) (2023)
- [Identifying latent distances with Finslerian geometry](https://mlsanity.com/paper/2212.10010/) (2022)
- [Hyperboloid GPLVM for Discovering Continuous Hierarchies via Nonparametric Estimation](https://mlsanity.com/paper/2410.16698/) (2024)
- [Atlas Gaussian processes on restricted domains and point clouds](https://mlsanity.com/paper/2511.15822/) (2025)
- [Intrinsic-Hybrid Latent Diffusion Models for Generative Modeling on Unknown Manifolds](https://mlsanity.com/paper/2608.04827/) (2026)

## Abstract (arXiv)

Decoders built on Gaussian processes (GPs) are enticing due to the marginalisation over the non-linear function space. Such models (also known as GP-LVMs) are often expensive and notoriously difficult to train in practice, but can be scaled using variational inference and inducing points. In this paper, we revisit active set approximations. We develop a new stochastic estimate of the log-marginal likelihood based on recently discovered links to cross-validation, and propose a computationally efficient approximation thereof. We demonstrate that the resulting stochastic active sets (SAS) approximation significantly improves the robustness of GP decoder training while reducing computational cost. The SAS-GP obtains more structure in the latent space, scales to many datapoints and learns better representations than variational autoencoders, which is rarely the case for GP decoders.