# Deep Frame Prediction for Video Coding

arXiv: https://arxiv.org/abs/1901.00062 · HTML: https://mlsanity.com/paper/1901.00062/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2018.

## Built on by (most cited first, 13 of 13)

- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation](https://mlsanity.com/paper/2207.08119/) (2022)
- [Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set](https://mlsanity.com/paper/2111.10302/) (2021)
- [Advancing Learned Video Compression with In-loop Frame Prediction](https://mlsanity.com/paper/2211.07004/) (2022)
- [JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation](https://mlsanity.com/paper/2206.04231/) (2022)
- [Conditional Video Generation for High-Efficiency Video Compression](https://mlsanity.com/paper/2507.15269/) (2025)
- [DyLoc: Dynamic Localization for Massive MIMO Using Predictive Recurrent Neural Networks](https://mlsanity.com/paper/2101.07848/) (2021)
- [Texture-aware Video Frame Interpolation](https://mlsanity.com/paper/2102.13520/) (2021)
- [H-VFI: Hierarchical Frame Interpolation for Videos with Large Motions](https://mlsanity.com/paper/2211.11309/) (2022)
- [Analysis of Latent-Space Motion for Collaborative Intelligence](https://mlsanity.com/paper/2102.04018/) (2021)
- [Neural Network based Inter bi-prediction Blending](https://mlsanity.com/paper/2202.03149/) (2022)
- [Benefits of Feature Extraction and Temporal Sequence Analysis for Video Frame Prediction: An Evaluation of Hybrid Deep Learning Models](https://mlsanity.com/paper/2508.00898/) (2025)
- [ANVIL: Accelerator-Native Video Interpolation via Codec Motion Vector Priors](https://mlsanity.com/paper/2603.26835/) (2026)

## Abstract (arXiv)

We propose a novel frame prediction method using a deep neural network (DNN), with the goal of improving video coding efficiency. The proposed DNN makes use of decoded frames, at both encoder and decoder, to predict textures of the current coding block. Unlike conventional inter-prediction, the proposed method does not require any motion information to be transferred between the encoder and the decoder. Still, both uni-directional and bi-directional prediction are possible using the proposed DNN, which is enabled by the use of the temporal index channel, in addition to color channels. In this study, we developed a jointly trained DNN for both uni- and bi- directional prediction, as well as separate networks for uni- and bi-directional prediction, and compared the efficacy of both approaches. The proposed DNNs were compared with the conventional motion-compensated prediction in the latest video coding standard, HEVC, in terms of BD-Bitrate. The experiments show that the proposed joint DNN (for both uni- and bi-directional prediction) reduces the luminance bitrate by about 4.4%, 2.4%, and 2.3% in the Low delay P, Low delay, and Random access configurations, respectively. In addition, using the separately trained DNNs brings further bit savings of about 0.3%-0.5%.