# A Temporally-Aware Interpolation Network for Video Frame Inpainting

arXiv: https://arxiv.org/abs/1803.07218 · HTML: https://mlsanity.com/paper/1803.07218/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2018.

## Built on by (most cited first, 6 of 6)

- [Facial Video-based Remote Physiological Measurement via Self-supervised Learning](https://mlsanity.com/paper/2210.15401/) (2022)
- [Interactive Conversational Head Generation](https://mlsanity.com/paper/2307.02090/) (2023)
- [Rethinking the Diffusion Models for Numerical Tabular Data Imputation from the Perspective of Wasserstein Gradient Flow](https://mlsanity.com/paper/2406.15762/) (2024)
- [TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image](https://mlsanity.com/paper/2609.04911/) (2026)
- [Sequential three-way decisions with a single hidden layer feedforward neural network](https://mlsanity.com/paper/2303.07589/) (2023)
- [Retrieval, Refinement, and Ranking for Text-to-Video Generation via Prompt Optimization and Test-Time Scaling](https://mlsanity.com/paper/2603.01509/) (2026)

## Abstract (arXiv)

We propose the first deep learning solution to video frame inpainting, a challenging instance of the general video inpainting problem with applications in video editing, manipulation, and forensics. Our task is less ambiguous than frame interpolation and video prediction because we have access to both the temporal context and a partial glimpse of the future, allowing us to better evaluate the quality of a model's predictions objectively. We devise a pipeline composed of two modules: a bidirectional video prediction module, and a temporally-aware frame interpolation module. The prediction module makes two intermediate predictions of the missing frames, one conditioned on the preceding frames and the other conditioned on the following frames, using a shared convolutional LSTM-based encoder-decoder. The interpolation module blends the intermediate predictions to form the final result. Specifically, it utilizes time information and hidden activations from the video prediction module to resolve disagreements between the predictions. Our experiments demonstrate that our approach produces more accurate and qualitatively satisfying results than a state-of-the-art video prediction method and many strong frame inpainting baselines.