# Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features

arXiv: https://arxiv.org/abs/2106.05960 · HTML: https://mlsanity.com/paper/2106.05960/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 18 of 18)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [The Gaussian Neural Process](https://mlsanity.com/paper/2101.03606/) (2021)
- [Avoiding pathologies in very deep networks](https://mlsanity.com/paper/1402.5836/) (2014)
- [Random Feature Expansions for Deep Gaussian Processes](https://mlsanity.com/paper/1610.04386/) (2016)
- [Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1806.05490/) (2018)
- [Heterogeneous Multi-output Gaussian Process Prediction](https://mlsanity.com/paper/1805.07633/) (2018)
- [Deep Compositional Spatial Models](https://mlsanity.com/paper/1906.02840/) (2019)
- [Deep Gaussian Processes with Importance-Weighted Variational Inference](https://mlsanity.com/paper/1905.05435/) (2019)
- [Implicit Posterior Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1910.11998/) (2019)
- [Efficient Amortised Bayesian Inference for Hierarchical and Nonlinear Dynamical Systems](https://mlsanity.com/paper/1905.12090/) (2019)
- [Black-Box Inference for Non-Linear Latent Force Models](https://mlsanity.com/paper/1906.09199/) (2019)
- [Constraining the Dynamics of Deep Probabilistic Models](https://mlsanity.com/paper/1802.05680/) (2018)
- [Fast Kernel Approximations for Latent Force Models and Convolved Multiple-Output Gaussian processes](https://mlsanity.com/paper/1805.07460/) (2018)

## Built on by (most cited first, 8 of 8)

- [A spectrum of physics-informed Gaussian processes for regression in engineering](https://mlsanity.com/paper/2309.10656/) (2023)
- [TransNet: Transferable Neural Networks for Partial Differential Equations](https://mlsanity.com/paper/2301.11701/) (2023)
- [Scalable Random Wavelet Features: Efficient Non-Stationary Kernel Approximation with Convergence Guarantees](https://mlsanity.com/paper/2602.00987/) (2026)
- [On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel](https://mlsanity.com/paper/2203.07411/) (2022)
- [Shallow and Deep Nonparametric Convolutions for Gaussian Processes](https://mlsanity.com/paper/2206.08972/) (2022)
- [Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learning](https://mlsanity.com/paper/2311.14828/) (2023)
- [Adaptive RKHS Fourier Features for Compositional Gaussian Process Models](https://mlsanity.com/paper/2407.01856/) (2024)
- [A Comparative Evaluation of Additive Separability Tests for Physics-Informed Machine Learning](https://mlsanity.com/paper/2312.09775/) (2023)

## Abstract (arXiv)

Effectively modeling phenomena present in highly nonlinear dynamical systems whilst also accurately quantifying uncertainty is a challenging task, which often requires problem-specific techniques. We present a novel, domain-agnostic approach to tackling this problem, using compositions of physics-informed random features, derived from ordinary differential equations. The architecture of our model leverages recent advances in approximate inference for deep Gaussian processes, such as layer-wise weight-space approximations which allow us to incorporate random Fourier features, and stochastic variational inference for approximate Bayesian inference. We provide evidence that our model is capable of capturing highly nonlinear behaviour in real-world multivariate time series data. In addition, we find that our approach achieves comparable performance to a number of other probabilistic models on benchmark regression tasks.