# Deep Generative Markov State Models

arXiv: https://arxiv.org/abs/1805.07601 · HTML: https://mlsanity.com/paper/1805.07601/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2018.

## Built on by (most cited first, 19 of 19)

- [Timewarp: Transferable Acceleration of Molecular Dynamics by Learning Time-Coarsened Dynamics](https://mlsanity.com/paper/2302.01170/) (2023)
- [Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoders](https://mlsanity.com/paper/2201.05136/) (2022)
- [Deeptime: a Python library for machine learning dynamical models from time series data](https://mlsanity.com/paper/2110.15013/) (2021)
- [Constructing Neural Network-Based Models for Simulating Dynamical Systems](https://mlsanity.com/paper/2111.01495/) (2021)
- [Simulate Time-integrated Coarse-grained Molecular Dynamics with Multi-Scale Graph Networks](https://mlsanity.com/paper/2204.10348/) (2022)
- [Dimensionally Consistent Learning with Buckingham Pi](https://mlsanity.com/paper/2202.04643/) (2022)
- [Implicit Transfer Operator Learning: Multiple Time-Resolution Surrogates for Molecular Dynamics](https://mlsanity.com/paper/2305.18046/) (2023)
- [Physics-aware, probabilistic model order reduction with guaranteed stability](https://mlsanity.com/paper/2101.05834/) (2021)
- [A Physics Informed Neural Network Approach to Solution and Identification of Biharmonic Equations of Elasticity](https://mlsanity.com/paper/2108.07243/) (2021)
- [Variational Inference for Continuous-Time Switching Dynamical Systems](https://mlsanity.com/paper/2109.14492/) (2021)
- [Designing Long-term Group Fair Policies in Dynamical Systems](https://mlsanity.com/paper/2311.12447/) (2023)
- [Exploring Diffusion and Flow Matching Under Generator Matching](https://mlsanity.com/paper/2412.11024/) (2024)
- [Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck](https://mlsanity.com/paper/2309.07200/) (2023)
- [On the Stochastic Stability of Deep Markov Models](https://mlsanity.com/paper/2111.04601/) (2021)
- [Machine Learning Applications to Computational Plasma Physics and Reduced-Order Plasma Modeling: A Perspective](https://mlsanity.com/paper/2409.02349/) (2024)
- [Beyond Ensembles: Simulating All-Atom Protein Dynamics in a Learned Latent Space](https://mlsanity.com/paper/2509.02196/) (2025)
- [Temporal Cardiovascular Dynamics for Improved PPG-Based Heart Rate Estimation](https://mlsanity.com/paper/2510.27297/) (2025)
- [Predicting the Energy Landscape of Stochastic Dynamical System via Physics-informed Self-supervised Learning](https://mlsanity.com/paper/2502.16828/) (2025)
- [Sampling Boltzmann distributions via normalizing flow approximation of transport maps](https://mlsanity.com/paper/2603.14258/) (2026)

## Abstract (arXiv)

We propose a deep generative Markov State Model (DeepGenMSM) learning framework for inference of metastable dynamical systems and prediction of trajectories. After unsupervised training on time series data, the model contains (i) a probabilistic encoder that maps from high-dimensional configuration space to a small-sized vector indicating the membership to metastable (long-lived) states, (ii) a Markov chain that governs the transitions between metastable states and facilitates analysis of the long-time dynamics, and (iii) a generative part that samples the conditional distribution of configurations in the next time step. The model can be operated in a recursive fashion to generate trajectories to predict the system evolution from a defined starting state and propose new configurations. The DeepGenMSM is demonstrated to provide accurate estimates of the long-time kinetics and generate valid distributions for molecular dynamics (MD) benchmark systems. Remarkably, we show that DeepGenMSMs are able to make long time-steps in molecular configuration space and generate physically realistic structures in regions that were not seen in training data.