# PDWN: Pyramid Deformable Warping Network for Video Interpolation

arXiv: https://arxiv.org/abs/2104.01517 · HTML: https://mlsanity.com/paper/2104.01517/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 4 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Deep multi-scale video prediction beyond mean square error](https://mlsanity.com/paper/1511.05440/) (2015)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [Bi-Directional Cascade Network for Perceptual Edge Detection](https://mlsanity.com/paper/1902.10903/) (2019)
- [Revisiting Adaptive Convolutions for Video Frame Interpolation](https://mlsanity.com/paper/2011.01280/) (2020)
- [Learning to Linearize Under Uncertainty](https://mlsanity.com/paper/1506.03011/) (2015)
- [Frame Interpolation with Multi-Scale Deep Loss Functions and Generative Adversarial Networks](https://mlsanity.com/paper/1711.06045/) (2017)

## Built on by (most cited first, 10 of 10)

- [FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation](https://mlsanity.com/paper/2207.08119/) (2022)
- [ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation](https://mlsanity.com/paper/2111.15483/) (2021)
- [Frame Interpolation with Consecutive Brownian Bridge Diffusion](https://mlsanity.com/paper/2405.05953/) (2024)
- [Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation](https://mlsanity.com/paper/2203.10291/) (2022)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [BVI-VFI: A Video Quality Database for Video Frame Interpolation](https://mlsanity.com/paper/2210.00823/) (2022)
- [H-VFI: Hierarchical Frame Interpolation for Videos with Large Motions](https://mlsanity.com/paper/2211.11309/) (2022)
- [Flow Guidance Deformable Compensation Network for Video Frame Interpolation](https://mlsanity.com/paper/2211.12117/) (2022)
- [EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation](https://mlsanity.com/paper/2505.08235/) (2025)
- [Motion-Aware Video Frame Interpolation](https://mlsanity.com/paper/2402.02892/) (2024)

## Abstract (arXiv)

Video interpolation aims to generate a non-existent intermediate frame given the past and future frames. Many state-of-the-art methods achieve promising results by estimating the optical flow between the known frames and then generating the backward flows between the middle frame and the known frames. However, these methods usually suffer from the inaccuracy of estimated optical flows and require additional models or information to compensate for flow estimation errors. Following the recent development in using deformable convolution (DConv) for video interpolation, we propose a light but effective model, called Pyramid Deformable Warping Network (PDWN). PDWN uses a pyramid structure to generate DConv offsets of the unknown middle frame with respect to the known frames through coarse-to-fine successive refinements. Cost volumes between warped features are calculated at every pyramid level to help the offset inference. At the finest scale, the two warped frames are adaptively blended to generate the middle frame. Lastly, a context enhancement network further enhances the contextual detail of the final output. Ablation studies demonstrate the effectiveness of the coarse-to-fine offset refinement, cost volumes, and DConv. Our method achieves better or on-par accuracy compared to state-of-the-art models on multiple datasets while the number of model parameters and the inference time are substantially less than previous models. Moreover, we present an extension of the proposed framework to use four input frames, which can achieve significant improvement over using only two input frames, with only a slight increase in the model size and inference time.