# A Log-likelihood Regularized KL Divergence for Video Prediction with A 3D Convolutional Variational Recurrent Network

arXiv: https://arxiv.org/abs/2012.06123 · HTML: https://mlsanity.com/paper/2012.06123/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2020.

## Built on by (most cited first, 3 of 3)

- [S-HR-VQVAE: Sequential Hierarchical Residual Learning Vector Quantized Variational Autoencoder for Video Prediction](https://mlsanity.com/paper/2307.06701/) (2023)
- [Predictive Coding Based Multiscale Network with Encoder-Decoder LSTM for Video Prediction](https://mlsanity.com/paper/2212.11642/) (2022)
- [Anti-aliasing Predictive Coding Network for Future Video Frame Prediction](https://mlsanity.com/paper/2301.05421/) (2023)

## Abstract (arXiv)

The use of latent variable models has shown to be a powerful tool for modeling probability distributions over sequences. In this paper, we introduce a new variational model that extends the recurrent network in two ways for the task of video frame prediction. First, we introduce 3D convolutions inside all modules including the recurrent model for future frame prediction, inputting and outputting a sequence of video frames at each timestep. This enables us to better exploit spatiotemporal information inside the variational recurrent model, allowing us to generate high-quality predictions. Second, we enhance the latent loss of the variational model by introducing a maximum likelihood estimate in addition to the KL divergence that is commonly used in variational models. This simple extension acts as a stronger regularizer in the variational autoencoder loss function and lets us obtain better results and generalizability. Experiments show that our model outperforms existing video prediction methods on several benchmarks while requiring fewer parameters.