# Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Data

arXiv: https://arxiv.org/abs/2208.12975 · HTML: https://mlsanity.com/paper/2208.12975/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 27 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Deep Kalman Filters](https://mlsanity.com/paper/1511.05121/) (2015)
- [Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data](https://mlsanity.com/paper/1605.06432/) (2016)
- [Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control](https://mlsanity.com/paper/2111.10992/) (2021)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)
- [A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning](https://mlsanity.com/paper/1710.05741/) (2017)
- [Stochastic Variational Deep Kernel Learning](https://mlsanity.com/paper/1611.00336/) (2016)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Manifold Gaussian Processes for Regression](https://mlsanity.com/paper/1402.5876/) (2014)
- [Probabilistic Recurrent State-Space Models](https://mlsanity.com/paper/1801.10395/) (2018)
- [Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks](https://mlsanity.com/paper/1707.02476/) (2017)
- [Learning and Querying Fast Generative Models for Reinforcement Learning](https://mlsanity.com/paper/1802.03006/) (2018)
- [From Pixels to Torques: Policy Learning with Deep Dynamical Models](https://mlsanity.com/paper/1502.02251/) (2015)
- [Bayesian operator inference for data-driven reduced-order modeling](https://mlsanity.com/paper/2204.10829/) (2022)
- [Adaptive Deep Kernel Learning](https://mlsanity.com/paper/1905.12131/) (2019)
- [Learning deep dynamical models from image pixels](https://mlsanity.com/paper/1410.7550/) (2014)
- [Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Models](https://mlsanity.com/paper/1510.02173/) (2015)

## Built on by (most cited first, 6 of 6)

- [Multi-fidelity reduced-order surrogate modeling](https://mlsanity.com/paper/2309.00325/) (2023)
- [VENI, VINDy, VICI: a generative reduced-order modeling framework with uncertainty quantification](https://mlsanity.com/paper/2405.20905/) (2024)
- [Gaussian process learning of nonlinear dynamics](https://mlsanity.com/paper/2312.12193/) (2023)
- [Recurrent Deep Kernel Learning of Dynamical Systems](https://mlsanity.com/paper/2405.19785/) (2024)
- [Towards scientific machine learning for granular material simulations -- challenges and opportunities](https://mlsanity.com/paper/2504.08766/) (2025)
- [Multi-Fidelity Delayed Acceptance: hierarchical MCMC sampling for Bayesian inverse problems combining multiple solvers through deep neural networks](https://mlsanity.com/paper/2512.16430/) (2025)

## Abstract (arXiv)

This work proposes a Stochastic Variational Deep Kernel Learning method for the data-driven discovery of low-dimensional dynamical models from high-dimensional noisy data. The framework is composed of an encoder that compresses high-dimensional measurements into low-dimensional state variables, and a latent dynamical model for the state variables that predicts the system evolution over time. The training of the proposed model is carried out in an unsupervised manner, i.e., not relying on labeled data. Our learning method is evaluated on the motion of a pendulum -- a well studied baseline for nonlinear model identification and control with continuous states and control inputs -- measured via high-dimensional noisy RGB images. Results show that the method can effectively denoise measurements, learn compact state representations and latent dynamical models, as well as identify and quantify modeling uncertainties.