# Probabilistic Recurrent State-Space Models

arXiv: https://arxiv.org/abs/1801.10395 · HTML: https://mlsanity.com/paper/1801.10395/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2018.

## Built on by (most cited first, 30 of 64)

- [Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation](https://mlsanity.com/paper/2410.13232/) (2024)
- [Video Prediction Models as Rewards for Reinforcement Learning](https://mlsanity.com/paper/2305.14343/) (2023)
- [Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)](https://mlsanity.com/paper/2505.16394/) (2025)
- [World Models via Policy-Guided Trajectory Diffusion](https://mlsanity.com/paper/2312.08533/) (2023)
- [Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series](https://mlsanity.com/paper/2301.11308/) (2023)
- [A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies](https://mlsanity.com/paper/2208.14197/) (2022)
- [LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios](https://mlsanity.com/paper/2310.08348/) (2023)
- [Unsupervised Representation Learning in Deep Reinforcement Learning: A Review](https://mlsanity.com/paper/2208.14226/) (2022)
- [RNN with Particle Flow for Probabilistic Spatio-temporal Forecasting](https://mlsanity.com/paper/2106.06064/) (2021)
- [Flow-based Recurrent Belief State Learning for POMDPs](https://mlsanity.com/paper/2205.11051/) (2022)
- [Variational multiple shooting for Bayesian ODEs with Gaussian processes](https://mlsanity.com/paper/2106.10905/) (2021)
- [An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Context](https://mlsanity.com/paper/2212.12735/) (2022)
- [High-Accuracy Model-Based Reinforcement Learning, a Survey](https://mlsanity.com/paper/2107.08241/) (2021)
- [On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2210.09256/) (2022)
- [R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models](https://mlsanity.com/paper/2409.14216/) (2024)
- [Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Series](https://mlsanity.com/paper/2410.05602/) (2024)
- [Structure-preserving Gaussian Process Dynamics](https://mlsanity.com/paper/2102.01606/) (2021)
- [GATSBI: Generative Agent-centric Spatio-temporal Object Interaction](https://mlsanity.com/paper/2104.04275/) (2021)
- [Recognition Models to Learn Dynamics from Partial Observations with Neural ODEs](https://mlsanity.com/paper/2205.12550/) (2022)
- [On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning](https://mlsanity.com/paper/2306.05637/) (2023)
- [Free-Form Variational Inference for Gaussian Process State-Space Models](https://mlsanity.com/paper/2302.09921/) (2023)
- [OPa-Ma: Text Guided Mamba for 360-degree Image Out-painting](https://mlsanity.com/paper/2407.10923/) (2024)
- [Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Data](https://mlsanity.com/paper/2208.12975/) (2022)
- [Output-Dependent Gaussian Process State-Space Model](https://mlsanity.com/paper/2212.07608/) (2022)
- [Learning to Play Atari in a World of Tokens](https://mlsanity.com/paper/2406.01361/) (2024)
- [A Temporal Variational Model for Story Generation](https://mlsanity.com/paper/2109.06807/) (2021)
- [Recurrent Neural Networks and Universal Approximation of Bayesian Filters](https://mlsanity.com/paper/2211.00335/) (2022)
- [Towards Flexibility and Interpretability of Gaussian Process State-Space Model](https://mlsanity.com/paper/2301.08843/) (2023)
- [Sequential Estimation of Gaussian Process-based Deep State-Space Models](https://mlsanity.com/paper/2301.12528/) (2023)
- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)

## Abstract (arXiv)

State-space models (SSMs) are a highly expressive model class for learning patterns in time series data and for system identification. Deterministic versions of SSMs (e.g. LSTMs) proved extremely successful in modeling complex time series data. Fully probabilistic SSMs, however, are often found hard to train, even for smaller problems. To overcome this limitation, we propose a novel model formulation and a scalable training algorithm based on doubly stochastic variational inference and Gaussian processes. In contrast to existing work, the proposed variational approximation allows one to fully capture the latent state temporal correlations. These correlations are the key to robust training. The effectiveness of the proposed PR-SSM is evaluated on a set of real-world benchmark datasets in comparison to state-of-the-art probabilistic model learning methods. Scalability and robustness are demonstrated on a high dimensional problem.