# Video Imagination from a Single Image with Transformation Generation

arXiv: https://arxiv.org/abs/1706.04124 · HTML: https://mlsanity.com/paper/1706.04124/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2017.

## Built on by (most cited first, 5 of 5)

- [FitVid: Overfitting in Pixel-Level Video Prediction](https://mlsanity.com/paper/2106.13195/) (2021)
- [Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction](https://mlsanity.com/paper/2103.04174/) (2021)
- [VPTR: Efficient Transformers for Video Prediction](https://mlsanity.com/paper/2203.15836/) (2022)
- [Video Prediction by Efficient Transformers](https://mlsanity.com/paper/2212.06026/) (2022)
- [CPNet: Cross-Parallel Network for Efficient Anomaly Detection](https://mlsanity.com/paper/2108.04454/) (2021)

## Abstract (arXiv)

In this work, we focus on a challenging task: synthesizing multiple imaginary videos given a single image. Major problems come from high dimensionality of pixel space and the ambiguity of potential motions. To overcome those problems, we propose a new framework that produce imaginary videos by transformation generation. The generated transformations are applied to the original image in a novel volumetric merge network to reconstruct frames in imaginary video. Through sampling different latent variables, our method can output different imaginary video samples. The framework is trained in an adversarial way with unsupervised learning. For evaluation, we propose a new assessment metric RIQA. In experiments, we test on 3 datasets varying from synthetic data to natural scene. Our framework achieves promising performance in image quality assessment. The visual inspection indicates that it can successfully generate diverse five-frame videos in acceptable perceptual quality.