# Gaussian Process Prior Variational Autoencoders

arXiv: https://arxiv.org/abs/1810.11738 · HTML: https://mlsanity.com/paper/1810.11738/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2018.

## Built on by (most cited first, 30 of 72)

- [SAITS: Self-Attention-based Imputation for Time Series](https://mlsanity.com/paper/2202.08516/) (2022)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering](https://mlsanity.com/paper/2201.00791/) (2022)
- [Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling](https://mlsanity.com/paper/2102.13156/) (2021)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Decoupling Local and Global Representations of Time Series](https://mlsanity.com/paper/2202.02262/) (2022)
- [Latent Neural ODEs with Sparse Bayesian Multiple Shooting](https://mlsanity.com/paper/2210.03466/) (2022)
- [Generative AI for Physical Layer Communications: A Survey](https://mlsanity.com/paper/2312.05594/) (2023)
- [Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation](https://mlsanity.com/paper/2107.03442/) (2021)
- [Markovian Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2207.05543/) (2022)
- [Latent SDEs on Homogeneous Spaces](https://mlsanity.com/paper/2306.16248/) (2023)
- [Scalable Inference in SDEs by Direct Matching of the Fokker-Planck-Kolmogorov Equation](https://mlsanity.com/paper/2110.15739/) (2021)
- [On Incorporating Inductive Biases into VAEs](https://mlsanity.com/paper/2106.13746/) (2021)
- [Revisiting Structured Variational Autoencoders](https://mlsanity.com/paper/2305.16543/) (2023)
- [A Probabilistic Approach to Wildfire Spread Prediction Using a Denoising Diffusion Surrogate Model](https://mlsanity.com/paper/2507.00761/) (2025)
- [Learning Conditional Variational Autoencoders with Missing Covariates](https://mlsanity.com/paper/2203.01218/) (2022)
- [Modulated Neural ODEs](https://mlsanity.com/paper/2302.13262/) (2023)
- [Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes](https://mlsanity.com/paper/2302.04534/) (2023)
- [Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data](https://mlsanity.com/paper/2310.03111/) (2023)
- [Coupled Variational Autoencoder](https://mlsanity.com/paper/2306.02565/) (2023)
- [Novel View Synthesis via Depth-guided Skip Connections](https://mlsanity.com/paper/2101.01619/) (2021)
- [On Disentanglement in Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2102.05507/) (2021)
- [Hamiltonian latent operators for content and motion disentanglement in image sequences](https://mlsanity.com/paper/2112.01641/) (2021)
- [Diffusion Transformer Captures Spatial-Temporal Dependencies: A Theory for Gaussian Process Data](https://mlsanity.com/paper/2407.16134/) (2024)
- [Half-VAE: An Encoder-Free VAE to Bypass Explicit Inverse Mapping](https://mlsanity.com/paper/2409.04140/) (2024)
- [Robust Simulation-Based Inference under Missing Data via Neural Processes](https://mlsanity.com/paper/2503.01287/) (2025)
- [Identifiable Feature Learning for Spatial Data with Nonlinear ICA](https://mlsanity.com/paper/2311.16849/) (2023)
- [Gaussian Measures Conditioned on Nonlinear Observations: Consistency, MAP Estimators, and Simulation](https://mlsanity.com/paper/2405.13149/) (2024)
- [Latent Space Energy-based Neural ODEs](https://mlsanity.com/paper/2409.03845/) (2024)

## Abstract (arXiv)

Variational autoencoders (VAE) are a powerful and widely-used class of models to learn complex data distributions in an unsupervised fashion. One important limitation of VAEs is the prior assumption that latent sample representations are independent and identically distributed. However, for many important datasets, such as time-series of images, this assumption is too strong: accounting for covariances between samples, such as those in time, can yield to a more appropriate model specification and improve performance in downstream tasks. In this work, we introduce a new model, the Gaussian Process (GP) Prior Variational Autoencoder (GPPVAE), to specifically address this issue. The GPPVAE aims to combine the power of VAEs with the ability to model correlations afforded by GP priors. To achieve efficient inference in this new class of models, we leverage structure in the covariance matrix, and introduce a new stochastic backpropagation strategy that allows for computing stochastic gradients in a distributed and low-memory fashion. We show that our method outperforms conditional VAEs (CVAEs) and an adaptation of standard VAEs in two image data applications.