# Physics-enhanced Gaussian Process Variational Autoencoder

arXiv: https://arxiv.org/abs/2305.09006 · HTML: https://mlsanity.com/paper/2305.09006/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 16 of 16)

- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [GP-VAE: Deep Probabilistic Time Series Imputation](https://mlsanity.com/paper/1907.04155/) (2019)
- [A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning](https://mlsanity.com/paper/1710.05741/) (2017)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [An Uncertain Future: Forecasting from Static Images using Variational Autoencoders](https://mlsanity.com/paper/1606.07873/) (2016)
- [Gaussian Process Prior Variational Autoencoders](https://mlsanity.com/paper/1810.11738/) (2018)
- [Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction](https://mlsanity.com/paper/1905.10866/) (2019)
- [Convolutional Gaussian Processes](https://mlsanity.com/paper/1709.01894/) (2017)
- [Clockwork Variational Autoencoders](https://mlsanity.com/paper/2102.09532/) (2021)
- [Sparse Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2010.10177/) (2020)
- [Variational Message Passing with Structured Inference Networks](https://mlsanity.com/paper/1803.05589/) (2018)
- [Learning Constrained Dynamics with Gauss Principle adhering Gaussian Processes](https://mlsanity.com/paper/2004.11238/) (2020)
- [Physics-informed Gaussian Process for Online Optimization of Particle Accelerators](https://mlsanity.com/paper/2009.03566/) (2020)
- [tvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder](https://mlsanity.com/paper/2006.04788/) (2020)
- [Visual anomaly detection in video by variational autoencoder](https://mlsanity.com/paper/2203.03872/) (2022)

## Built on by (most cited first, 2 of 2)

- [A spectrum of physics-informed Gaussian processes for regression in engineering](https://mlsanity.com/paper/2309.10656/) (2023)
- [Stability Enhanced Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2604.09331/) (2026)

## Abstract (arXiv)

Variational autoencoders allow to learn a lower-dimensional latent space based on high-dimensional input/output data. Using video clips as input data, the encoder may be used to describe the movement of an object in the video without ground truth data (unsupervised learning). Even though the object's dynamics is typically based on first principles, this prior knowledge is mostly ignored in the existing literature. Thus, we propose a physics-enhanced variational autoencoder that places a physical-enhanced Gaussian process prior on the latent dynamics to improve the efficiency of the variational autoencoder and to allow physically correct predictions. The physical prior knowledge expressed as linear dynamical system is here reflected by the Green's function and included in the kernel function of the Gaussian process. The benefits of the proposed approach are highlighted in a simulation with an oscillating particle.