# Video Prediction with Appearance and Motion Conditions

arXiv: https://arxiv.org/abs/1807.02635 · HTML: https://mlsanity.com/paper/1807.02635/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2018.

## Built on by (most cited first, 18 of 18)

- [Latent Image Animator: Learning to Animate Images via Latent Space Navigation](https://mlsanity.com/paper/2203.09043/) (2022)
- [Reinforcement Learning with Action-Free Pre-Training from Videos](https://mlsanity.com/paper/2203.13880/) (2022)
- [Predicting Video with VQVAE](https://mlsanity.com/paper/2103.01950/) (2021)
- [On the Content Bias in Fr'echet Video Distance](https://mlsanity.com/paper/2404.12391/) (2024)
- [LEO: Generative Latent Image Animator for Human Video Synthesis](https://mlsanity.com/paper/2305.03989/) (2023)
- [HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator](https://mlsanity.com/paper/2209.07143/) (2022)
- [InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation](https://mlsanity.com/paper/2101.03049/) (2021)
- [Video Prediction by Efficient Transformers](https://mlsanity.com/paper/2212.06026/) (2022)
- [A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approaches](https://mlsanity.com/paper/2401.14718/) (2024)
- [Facial Expression Video Generation Based-On Spatio-temporal Convolutional GAN: FEV-GAN](https://mlsanity.com/paper/2210.11182/) (2022)
- [Artificial Intelligence for Biomedical Video Generation](https://mlsanity.com/paper/2411.07619/) (2024)
- [Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling](https://mlsanity.com/paper/2504.05537/) (2025)
- [Dual-MTGAN: Stochastic and Deterministic Motion Transfer for Image-to-Video Synthesis](https://mlsanity.com/paper/2102.13329/) (2021)
- [Continuous-Time Video Generation via Learning Motion Dynamics with Neural ODE](https://mlsanity.com/paper/2112.10960/) (2021)
- [V3GAN: Decomposing Background, Foreground and Motion for Video Generation](https://mlsanity.com/paper/2203.14074/) (2022)
- [Modeling the geospatial evolution of COVID-19 using spatio-temporal convolutional sequence-to-sequence neural networks](https://mlsanity.com/paper/2105.02752/) (2021)
- [Ebola Optimization Search Algorithm (EOSA): A new metaheuristic algorithm based on the propagation model of Ebola virus disease](https://mlsanity.com/paper/2106.01416/) (2021)
- [RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects](https://mlsanity.com/paper/2205.06975/) (2022)

## Abstract (arXiv)

Video prediction aims to generate realistic future frames by learning dynamic visual patterns. One fundamental challenge is to deal with future uncertainty: How should a model behave when there are multiple correct, equally probable future? We propose an Appearance-Motion Conditional GAN to address this challenge. We provide appearance and motion information as conditions that specify how the future may look like, reducing the level of uncertainty. Our model consists of a generator, two discriminators taking charge of appearance and motion pathways, and a perceptual ranking module that encourages videos of similar conditions to look similar. To train our model, we develop a novel conditioning scheme that consists of different combinations of appearance and motion conditions. We evaluate our model using facial expression and human action datasets and report favorable results compared to existing methods.