# Structure-preserving Gaussian Process Dynamics

arXiv: https://arxiv.org/abs/2102.01606 · HTML: https://mlsanity.com/paper/2102.01606/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 56 references that are in the map.
- First posted on arXiv 2 Feb 2021.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Probabilistic Recurrent State-Space Models](https://mlsanity.com/paper/1801.10395/) (2018)
- [Learning unknown ODE models with Gaussian processes](https://mlsanity.com/paper/1803.04303/) (2018)
- [Variational Gaussian Process State-Space Models](https://mlsanity.com/paper/1406.4905/) (2014)
- [Variational Integrator Networks for Physically Structured Embeddings](https://mlsanity.com/paper/1910.09349/) (2019)
- [Actively Learning Gaussian Process Dynamics](https://mlsanity.com/paper/1911.09946/) (2019)
- [Overcoming Mean-Field Approximations in Recurrent Gaussian Process Models](https://mlsanity.com/paper/1906.05828/) (2019)
- [Computationally Efficient Bayesian Learning of Gaussian Process State Space Models](https://mlsanity.com/paper/1506.02267/) (2015)
- [Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEs](https://mlsanity.com/paper/1804.04378/) (2018)
- [Symplectic Gaussian Process Regression of Hamiltonian Flow Maps](https://mlsanity.com/paper/2009.05569/) (2020)
- [Learning Constrained Dynamics with Gauss Principle adhering Gaussian Processes](https://mlsanity.com/paper/2004.11238/) (2020)
- [An implicit function learning approach for parametric modal regression](https://mlsanity.com/paper/2002.06195/) (2020)
- [Differentiable Implicit Layers](https://mlsanity.com/paper/2010.07078/) (2020)
- [Learning nonparametric differential equations with operator-valued kernels and gradient matching](https://mlsanity.com/paper/1411.5172/) (2014)
- [Deep recurrent Gaussian process with variational Sparse Spectrum approximation](https://mlsanity.com/paper/1909.13743/) (2019)

## Built on by (most cited first, 9 of 9)

- [Variational multiple shooting for Bayesian ODEs with Gaussian processes](https://mlsanity.com/paper/2106.10905/) (2021)
- [Recognition Models to Learn Dynamics from Partial Observations with Neural ODEs](https://mlsanity.com/paper/2205.12550/) (2022)
- [Learning Energy Conserving Dynamics Efficiently with Hamiltonian Gaussian Processes](https://mlsanity.com/paper/2303.01925/) (2023)
- [Learning Interacting Dynamical Systems with Latent Gaussian Process ODEs](https://mlsanity.com/paper/2205.11894/) (2022)
- [Structure-Preserving Learning Using Gaussian Processes and Variational Integrators](https://mlsanity.com/paper/2112.05451/) (2021)
- [The Past Does Matter: Correlation of Subsequent States in Trajectory Predictions of Gaussian Process Models](https://mlsanity.com/paper/2211.11103/) (2022)
- [Data-efficient Kernel Methods for Learning Hamiltonian Systems](https://mlsanity.com/paper/2509.17154/) (2025)
- [Learning Dynamics from Input-Output Data with Hamiltonian Gaussian Processes](https://mlsanity.com/paper/2511.05330/) (2025)
- [Structure-Preserving Gaussian Processes Via Discrete Euler-Lagrange Equations](https://mlsanity.com/paper/2605.06246/) (2026)

## Abstract (arXiv)

Most physical processes posses structural properties such as constant energies, volumes, and other invariants over time. When learning models of such dynamical systems, it is critical to respect these invariants to ensure accurate predictions and physically meaningful behavior. Strikingly, state-of-the-art methods in Gaussian process (GP) dynamics model learning are not addressing this issue. On the other hand, classical numerical integrators are specifically designed to preserve these crucial properties through time. We propose to combine the advantages of GPs as function approximators with structure preserving numerical integrators for dynamical systems, such as Runge-Kutta methods. These integrators assume access to the ground truth dynamics and require evaluations of intermediate and future time steps that are unknown in a learning-based scenario. This makes direct inference of the GP dynamics, with embedded numerical scheme, intractable. Our key technical contribution is the evaluation of the implicitly defined Runge-Kutta transition probability. In a nutshell, we introduce an implicit layer for GP regression, which is embedded into a variational inference-based model learning scheme.