# Stochastic Dynamics for Video Infilling

arXiv: https://arxiv.org/abs/1809.00263 · HTML: https://mlsanity.com/paper/1809.00263/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Sep 2018.

## Built on by (most cited first, 8 of 8)

- [MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation](https://mlsanity.com/paper/2205.09853/) (2022)
- [Diffusion Models for Video Prediction and Infilling](https://mlsanity.com/paper/2206.07696/) (2022)
- [Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning](https://mlsanity.com/paper/2104.00924/) (2021)
- [Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation](https://mlsanity.com/paper/2211.12824/) (2022)
- [Video Diffusion Models with Local-Global Context Guidance](https://mlsanity.com/paper/2306.02562/) (2023)
- [A unified model for continuous conditional video prediction](https://mlsanity.com/paper/2210.05810/) (2022)
- [Physical Context and Timing Aware Sequence Generating GANs](https://mlsanity.com/paper/2110.04077/) (2021)
- [Video Generation with Learned Action Prior](https://mlsanity.com/paper/2406.14436/) (2024)

## Abstract (arXiv)

In this paper, we introduce a stochastic dynamics video infilling (SDVI) framework to generate frames between long intervals in a video. Our task differs from video interpolation which aims to produce transitional frames for a short interval between every two frames and increase the temporal resolution. Our task, namely video infilling, however, aims to infill long intervals with plausible frame sequences. Our framework models the infilling as a constrained stochastic generation process and sequentially samples dynamics from the inferred distribution. SDVI consists of two parts: (1) a bi-directional constraint propagation module to guarantee the spatial-temporal coherence among frames, (2) a stochastic sampling process to generate dynamics from the inferred distributions. Experimental results show that SDVI can generate clear frame sequences with varying contents. Moreover, motions in the generated sequence are realistic and able to transfer smoothly from the given start frame to the terminal frame. Our project site is https://xharlie.github.io/projects/project_sites/SDVI/video_results.html