# Markov Decision Process for Video Generation

arXiv: https://arxiv.org/abs/1909.12400 · HTML: https://mlsanity.com/paper/1909.12400/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2019.

## Built on by (most cited first, 14 of 14)

- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation](https://mlsanity.com/paper/2205.09853/) (2022)
- [Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks](https://mlsanity.com/paper/2202.10571/) (2022)
- [Video Probabilistic Diffusion Models in Projected Latent Space](https://mlsanity.com/paper/2302.07685/) (2023)
- [A Good Image Generator Is What You Need for High-Resolution Video Synthesis](https://mlsanity.com/paper/2104.15069/) (2021)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN](https://mlsanity.com/paper/2107.07224/) (2021)
- [Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning](https://mlsanity.com/paper/2203.02573/) (2022)
- [Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs](https://mlsanity.com/paper/2308.13812/) (2023)
- [3D-Aware Video Generation](https://mlsanity.com/paper/2206.14797/) (2022)
- [MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation](https://mlsanity.com/paper/2502.12632/) (2025)
- [MotionVideoGAN: A Novel Video Generator Based on the Motion Space Learned from Image Pairs](https://mlsanity.com/paper/2303.02906/) (2023)
- [Video Generation Beyond a Single Clip](https://mlsanity.com/paper/2304.07483/) (2023)

## Abstract (arXiv)

We identify two pathological cases of temporal inconsistencies in video generation: video freezing and video looping. To better quantify the temporal diversity, we propose a class of complementary metrics that are effective, easy to implement, data agnostic, and interpretable. Further, we observe that current state-of-the-art models are trained on video samples of fixed length thereby inhibiting long-term modeling. To address this, we reformulate the problem of video generation as a Markov Decision Process (MDP). The underlying idea is to represent motion as a stochastic process with an infinite forecast horizon to overcome the fixed length limitation and to mitigate the presence of temporal artifacts. We show that our formulation is easy to integrate into the state-of-the-art MoCoGAN framework. Our experiments on the Human Actions and UCF-101 datasets demonstrate that our MDP-based model is more memory efficient and improves the video quality both in terms of the new and established metrics.