# FG-DFPN: Flow Guided Deformable Frame Prediction Network

arXiv: https://arxiv.org/abs/2503.11343 · HTML: https://mlsanity.com/paper/2503.11343/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [A Review on Deep Learning Techniques for Video Prediction](https://mlsanity.com/paper/2004.05214/) (2020)
- [Learning to Generate Long-term Future via Hierarchical Prediction](https://mlsanity.com/paper/1704.05831/) (2017)
- [SDCNet: Video Prediction Using Spatially-Displaced Convolution](https://mlsanity.com/paper/1811.00684/) (2018)
- [On the Computation of PSNR for a Set of Images or Video](https://mlsanity.com/paper/2104.14868/) (2021)
- [Multi-Scale Deformable Alignment and Content-Adaptive Inference for Flexible-Rate Bi-Directional Video Compression](https://mlsanity.com/paper/2306.16544/) (2023)
- [Motion-Adaptive Inference for Flexible Learned B-Frame Compression](https://mlsanity.com/paper/2402.08550/) (2024)
- [Can Learned Frame-Prediction Compete with Block-Motion Compensation for Video Coding?](https://mlsanity.com/paper/2007.08922/) (2020)
- [Effect of Architectures and Training Methods on the Performance of Learned Video Frame Prediction](https://mlsanity.com/paper/2008.06106/) (2020)
- [DFPN: Deformable Frame Prediction Network](https://mlsanity.com/paper/2105.12794/) (2021)
- [Everything is a Video: Unifying Modalities through Next-Frame Prediction](https://mlsanity.com/paper/2411.10503/) (2024)

## Abstract (arXiv)

Video frame prediction remains a fundamental challenge in computer vision with direct implications for autonomous systems, video compression, and media synthesis. We present FG-DFPN, a novel architecture that harnesses the synergy between optical flow estimation and deformable convolutions to model complex spatio-temporal dynamics. By guiding deformable sampling with motion cues, our approach addresses the limitations of fixed-kernel networks when handling diverse motion patterns. The multi-scale design enables FG-DFPN to simultaneously capture global scene transformations and local object movements with remarkable precision. Our experiments demonstrate that FG-DFPN achieves state-of-the-art performance on eight diverse MPEG test sequences, outperforming existing methods by 1dB PSNR while maintaining competitive inference speeds. The integration of motion cues with adaptive geometric transformations makes FG-DFPN a promising solution for next-generation video processing systems that require high-fidelity temporal predictions. The model and instructions to reproduce our results will be released at: https://github.com/KUIS-AI-Tekalp-Research Group/frame-prediction