# Sequential Neural Models with Stochastic Layers

arXiv: https://arxiv.org/abs/1605.07571 · HTML: https://mlsanity.com/paper/1605.07571/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 May 2016.

## Built on by (most cited first, 30 of 115)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [Modeling Irregular Time Series with Continuous Recurrent Units](https://mlsanity.com/paper/2111.11344/) (2021)
- [Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion](https://mlsanity.com/paper/2202.04770/) (2022)
- [RAVE: A variational autoencoder for fast and high-quality neural audio synthesis](https://mlsanity.com/paper/2111.05011/) (2021)
- [SocialVAE: Human Trajectory Prediction using Timewise Latents](https://mlsanity.com/paper/2203.08207/) (2022)
- [Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling](https://mlsanity.com/paper/2102.13156/) (2021)
- [Criteria for Classifying Forecasting Methods](https://mlsanity.com/paper/2212.03523/) (2022)
- [Causal Recurrent Variational Autoencoder for Medical Time Series Generation](https://mlsanity.com/paper/2301.06574/) (2023)
- [Noisy Recurrent Neural Networks](https://mlsanity.com/paper/2102.04877/) (2021)
- [Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder](https://mlsanity.com/paper/2102.01331/) (2021)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems](https://mlsanity.com/paper/2207.02542/) (2022)
- [Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecasting](https://mlsanity.com/paper/2307.08243/) (2023)
- [Constructing Neural Network-Based Models for Simulating Dynamical Systems](https://mlsanity.com/paper/2111.01495/) (2021)
- [Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2102.00397/) (2021)
- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [Model-Based Reinforcement Learning via Latent-Space Collocation](https://mlsanity.com/paper/2106.13229/) (2021)
- [Bayesian Attention Belief Networks](https://mlsanity.com/paper/2106.05251/) (2021)
- [Learning Belief Representations for Imitation Learning in POMDPs](https://mlsanity.com/paper/1906.09510/) (2019)
- [CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Process](https://mlsanity.com/paper/2401.14535/) (2024)
- [Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction](https://mlsanity.com/paper/2206.01899/) (2022)
- [Generative Recursive Reasoning](https://mlsanity.com/paper/2605.19376/) (2026)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos](https://mlsanity.com/paper/2409.02638/) (2024)
- [CAMul: Calibrated and Accurate Multi-view Time-Series Forecasting](https://mlsanity.com/paper/2109.07438/) (2021)
- [Data-to-text Generation with Variational Sequential Planning](https://mlsanity.com/paper/2202.13756/) (2022)
- [RNN with Particle Flow for Probabilistic Spatio-temporal Forecasting](https://mlsanity.com/paper/2106.06064/) (2021)
- [HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAE](https://mlsanity.com/paper/2204.01565/) (2022)
- [Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models](https://mlsanity.com/paper/2101.07046/) (2021)

## Abstract (arXiv)

How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The clear separation of deterministic and stochastic layers allows a structured variational inference network to track the factorization of the model's posterior distribution. By retaining both the nonlinear recursive structure of a recurrent neural network and averaging over the uncertainty in a latent path, like a state space model, we improve the state of the art results on the Blizzard and TIMIT speech modeling data sets by a large margin, while achieving comparable performances to competing methods on polyphonic music modeling.