# Sequential Estimation of Gaussian Process-based Deep State-Space Models

arXiv: https://arxiv.org/abs/2301.12528 · HTML: https://mlsanity.com/paper/2301.12528/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 29 Jan 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Probabilistic Recurrent State-Space Models](https://mlsanity.com/paper/1801.10395/) (2018)
- [Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks](https://mlsanity.com/paper/1301.3853/) (2013)
- [Variational Gaussian Process State-Space Models](https://mlsanity.com/paper/1406.4905/) (2014)
- [Very Deep Convolutional Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/1610.03022/) (2016)
- [Random Feature Expansions for Deep Gaussian Processes](https://mlsanity.com/paper/1610.04386/) (2016)
- [Identification of Gaussian Process State Space Models](https://mlsanity.com/paper/1705.10888/) (2017)
- [Deep State Space Models for Nonlinear System Identification](https://mlsanity.com/paper/2003.14162/) (2020)
- [Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.02720/) (2017)
- [Computationally Efficient Bayesian Learning of Gaussian Process State Space Models](https://mlsanity.com/paper/1506.02267/) (2015)
- [A flexible state space model for learning nonlinear dynamical systems](https://mlsanity.com/paper/1603.05486/) (2016)
- [Deep Learning Object Detection Methods for Ecological Camera Trap Data](https://mlsanity.com/paper/1803.10842/) (2018)
- [Weakly-Supervised Deep Learning of Heat Transport via Physics Informed Loss](https://mlsanity.com/paper/1807.11374/) (2018)
- [Deep State-Space Gaussian Processes](https://mlsanity.com/paper/2008.04733/) (2020)
- [Latent variable modeling with random features](https://mlsanity.com/paper/2006.11145/) (2020)
- [Prediction with Approximated Gaussian Process Dynamical Models](https://mlsanity.com/paper/2006.14551/) (2020)

## Built on by (most cited first, 5 of 5)

- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)
- [Dynamic Online Ensembles of Basis Expansions](https://mlsanity.com/paper/2405.01365/) (2024)
- [Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems](https://mlsanity.com/paper/2503.18309/) (2025)
- [Recursive Gaussian Process State Space Model](https://mlsanity.com/paper/2411.14679/) (2024)
- [Simulation-Free Learning of GP-SDEs from Irregular Observations](https://mlsanity.com/paper/2609.33112/) (2026)

## Abstract (arXiv)

We consider the problem of sequential estimation of the unknowns of state-space and deep state-space models that include estimation of functions and latent processes of the models. The proposed approach relies on Gaussian and deep Gaussian processes that are implemented via random feature-based Gaussian processes. In these models, we have two sets of unknowns, highly nonlinear unknowns (the values of the latent processes) and conditionally linear unknowns (the constant parameters of the random feature-based Gaussian processes). We present a method based on particle filtering where the parameters of the random feature-based Gaussian processes are integrated out in obtaining the predictive density of the states and do not need particles. We also propose an ensemble version of the method, with each member of the ensemble having its own set of features. With several experiments, we show that the method can track the latent processes up to a scale and rotation.