# Dynamical Variational Autoencoders: A Comprehensive Review

arXiv: https://arxiv.org/abs/2008.12595 · HTML: https://mlsanity.com/paper/2008.12595/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2020.

## Built on by (most cited first, 30 of 128)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Score-based Data Assimilation](https://mlsanity.com/paper/2306.10574/) (2023)
- [Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement](https://mlsanity.com/paper/2301.03028/) (2023)
- [Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs](https://mlsanity.com/paper/2310.02619/) (2023)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems](https://mlsanity.com/paper/2207.02542/) (2022)
- [Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecasting](https://mlsanity.com/paper/2307.08243/) (2023)
- [Amortized Variational Inference: When and Why?](https://mlsanity.com/paper/2307.11018/) (2023)
- [Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders](https://mlsanity.com/paper/2106.12271/) (2021)
- [Multifactor Sequential Disentanglement via Structured Koopman Autoencoders](https://mlsanity.com/paper/2303.17264/) (2023)
- [Temporally Disentangled Representation Learning under Unknown Nonstationarity](https://mlsanity.com/paper/2310.18615/) (2023)
- [Generative learning for nonlinear dynamics](https://mlsanity.com/paper/2311.04128/) (2023)
- [DANSE: Data-driven Non-linear State Estimation of Model-free Process in Unsupervised Learning Setup](https://mlsanity.com/paper/2306.03897/) (2023)
- [HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAE](https://mlsanity.com/paper/2204.01565/) (2022)
- [Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency](https://mlsanity.com/paper/2205.13476/) (2022)
- [Generative Models in Decision Making: A Survey](https://mlsanity.com/paper/2502.17100/) (2025)
- [Conditional Generation of Medical Time Series for Extrapolation to Underrepresented Populations](https://mlsanity.com/paper/2201.08186/) (2022)
- [Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics](https://mlsanity.com/paper/2212.07892/) (2022)
- [On latent dynamics learning in nonlinear reduced order modeling](https://mlsanity.com/paper/2408.15183/) (2024)
- [Flow Matching Meets Biology and Life Science: A Survey](https://mlsanity.com/paper/2507.17731/) (2025)
- [Physics-guided Deep Markov Models for Learning Nonlinear Dynamical Systems with Uncertainty](https://mlsanity.com/paper/2110.08607/) (2021)
- [Interpretable learning of effective dynamics for multiscale systems](https://mlsanity.com/paper/2309.05812/) (2023)
- [Exploring the limits of Hierarchical World Models in Reinforcement Learning](https://mlsanity.com/paper/2406.00483/) (2024)
- [Exploring the Latent Space of Autoencoders with Interventional Assays](https://mlsanity.com/paper/2106.16091/) (2021)
- [An Overview on Generative AI at Scale with Edge-Cloud Computing](https://mlsanity.com/paper/2306.17170/) (2023)
- [Generative AI and Process Systems Engineering: The Next Frontier](https://mlsanity.com/paper/2402.10977/) (2024)
- [MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization](https://mlsanity.com/paper/2507.07997/) (2025)
- [Disentangled Generative Models for Robust Prediction of System Dynamics](https://mlsanity.com/paper/2108.11684/) (2021)

## Abstract (arXiv)

Variational autoencoders (VAEs) are powerful deep generative models widely used to represent high-dimensional complex data through a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, the input data vectors are processed independently. Recently, a series of papers have presented different extensions of the VAE to process sequential data, which model not only the latent space but also the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks or state-space models. In this paper, we perform a literature review of these models. We introduce and discuss a general class of models, called dynamical variational autoencoders (DVAEs), which encompasses a large subset of these temporal VAE extensions. Then, we present in detail seven recently proposed DVAE models, with an aim to homogenize the notations and presentation lines, as well as to relate these models with existing classical temporal models. We have reimplemented those seven DVAE models and present the results of an experimental benchmark conducted on the speech analysis-resynthesis task (the PyTorch code is made publicly available). The paper concludes with a discussion on important issues concerning the DVAE class of models and future research guidelines.