# Unified Arbitrary-Time Video Frame Interpolation and Prediction

arXiv: https://arxiv.org/abs/2503.02316 · HTML: https://mlsanity.com/paper/2503.02316/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 18 of 18)

- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Domain Generalization for Object Recognition with Multi-task Autoencoders](https://mlsanity.com/paper/1508.07680/) (2015)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [DeepStereo: Learning to Predict New Views from the World's Imagery](https://mlsanity.com/paper/1506.06825/) (2015)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [Video Frame Interpolation Transformer](https://mlsanity.com/paper/2111.13817/) (2021)
- [A Review on Deep Learning Techniques for Video Prediction](https://mlsanity.com/paper/2004.05214/) (2020)
- [Improved Conditional VRNNs for Video Prediction](https://mlsanity.com/paper/1904.12165/) (2019)
- [Asymmetric Bilateral Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2108.06815/) (2021)
- [A Dynamic Multi-Scale Voxel Flow Network for Video Prediction](https://mlsanity.com/paper/2303.09875/) (2023)
- [A Unified Pyramid Recurrent Network for Video Frame Interpolation](https://mlsanity.com/paper/2211.03456/) (2022)
- [Optimizing Video Prediction via Video Frame Interpolation](https://mlsanity.com/paper/2206.13454/) (2022)
- [Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation](https://mlsanity.com/paper/2203.10291/) (2022)
- [Neighbor Correspondence Matching for Flow-based Video Frame Synthesis](https://mlsanity.com/paper/2207.06763/) (2022)

## Built on by (most cited first, 2 of 2)

- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [RocSync: Millisecond-Accurate Temporal Synchronization for Heterogeneous Camera Systems](https://mlsanity.com/paper/2511.14948/) (2025)

## Abstract (arXiv)

Video frame interpolation and prediction aim to synthesize frames in-between and subsequent to existing frames, respectively. Despite being closely-related, these two tasks are traditionally studied with different model architectures, or same architecture but individually trained weights. Furthermore, while arbitrary-time interpolation has been extensively studied, the value of arbitrary-time prediction has been largely overlooked. In this work, we present uniVIP - unified arbitrary-time Video Interpolation and Prediction. Technically, we firstly extend an interpolation-only network for arbitrary-time interpolation and prediction, with a special input channel for task (interpolation or prediction) encoding. Then, we show how to train a unified model on common triplet frames. Our uniVIP provides competitive results for video interpolation, and outperforms existing state-of-the-arts for video prediction. Codes will be available at: https://github.com/srcn-ivl/uniVIP