# IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation

arXiv: https://arxiv.org/abs/2205.14620 · HTML: https://mlsanity.com/paper/2205.14620/

## Key facts

- Cited by 109 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 58 references that are in the map.
- First posted on arXiv 29 May 2022.

## Builds on (most cited first, 30 of 38)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [View Synthesis by Appearance Flow](https://mlsanity.com/paper/1605.03557/) (2016)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation](https://mlsanity.com/paper/2007.12622/) (2020)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [Asymmetric Bilateral Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2108.06815/) (2021)
- [FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation](https://mlsanity.com/paper/2012.08512/) (2020)
- [Deep Animation Video Interpolation in the Wild](https://mlsanity.com/paper/2104.02495/) (2021)
- [DDFlow: Learning Optical Flow with Unlabeled Data Distillation](https://mlsanity.com/paper/1902.09145/) (2019)

## Built on by (most cited first, 30 of 109)

- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2303.00440/) (2023)
- [LDMVFI: Video Frame Interpolation with Latent Diffusion Models](https://mlsanity.com/paper/2303.09508/) (2023)
- [VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation](https://mlsanity.com/paper/2309.00398/) (2023)
- [Video Interpolation with Diffusion Models](https://mlsanity.com/paper/2404.01203/) (2024)
- [A Dynamic Multi-Scale Voxel Flow Network for Video Prediction](https://mlsanity.com/paper/2303.09875/) (2023)
- [A Unified Pyramid Recurrent Network for Video Frame Interpolation](https://mlsanity.com/paper/2211.03456/) (2022)
- [FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation](https://mlsanity.com/paper/2207.08119/) (2022)
- [TransFlow: Transformer as Flow Learner](https://mlsanity.com/paper/2304.11523/) (2023)
- [TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation](https://mlsanity.com/paper/2410.04221/) (2024)
- [NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results](https://mlsanity.com/paper/2404.11313/) (2024)
- [Sparse Global Matching for Video Frame Interpolation with Large Motion](https://mlsanity.com/paper/2404.06913/) (2024)
- [A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild](https://mlsanity.com/paper/2301.05191/) (2023)
- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [Event-Based Video Frame Interpolation With Cross-Modal Asymmetric Bidirectional Motion Fields](https://mlsanity.com/paper/2502.13716/) (2025)
- [Framer: Interactive Frame Interpolation](https://mlsanity.com/paper/2410.18978/) (2024)
- [Generative Inbetweening through Frame-wise Conditions-Driven Video Generation](https://mlsanity.com/paper/2412.11755/) (2024)
- [Perception-Oriented Video Frame Interpolation via Asymmetric Blending](https://mlsanity.com/paper/2404.06692/) (2024)
- [Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolation](https://mlsanity.com/paper/2412.07761/) (2024)
- [VFIMamba: Video Frame Interpolation with State Space Models](https://mlsanity.com/paper/2407.02315/) (2024)
- [ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler](https://mlsanity.com/paper/2410.05651/) (2024)
- [Frame Interpolation with Consecutive Brownian Bridge Diffusion](https://mlsanity.com/paper/2405.05953/) (2024)
- [EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation](https://mlsanity.com/paper/2503.15831/) (2025)
- [Generalizable Implicit Motion Modeling for Video Frame Interpolation](https://mlsanity.com/paper/2407.08680/) (2024)
- [DreamMover: Leveraging the Prior of Diffusion Models for Image Interpolation with Large Motion](https://mlsanity.com/paper/2409.09605/) (2024)
- [Motion-aware Latent Diffusion Models for Video Frame Interpolation](https://mlsanity.com/paper/2404.13534/) (2024)
- [BiM-VFI: Bidirectional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motions](https://mlsanity.com/paper/2412.11365/) (2024)
- [Motion-Aware Generative Frame Interpolation](https://mlsanity.com/paper/2501.03699/) (2025)
- [IBVC: Interpolation-driven B-frame Video Compression](https://mlsanity.com/paper/2309.13835/) (2023)
- [SfM-Free 3D Gaussian Splatting via Hierarchical Training](https://mlsanity.com/paper/2412.01553/) (2024)

## Abstract (arXiv)

Prevailing video frame interpolation algorithms, that generate the intermediate frames from consecutive inputs, typically rely on complex model architectures with heavy parameters or large delay, hindering them from diverse real-time applications. In this work, we devise an efficient encoder-decoder based network, termed IFRNet, for fast intermediate frame synthesizing. It first extracts pyramid features from given inputs, and then refines the bilateral intermediate flow fields together with a powerful intermediate feature until generating the desired output. The gradually refined intermediate feature can not only facilitate intermediate flow estimation, but also compensate for contextual details, making IFRNet do not need additional synthesis or refinement module. To fully release its potential, we further propose a novel task-oriented optical flow distillation loss to focus on learning the useful teacher knowledge towards frame synthesizing. Meanwhile, a new geometry consistency regularization term is imposed on the gradually refined intermediate features to keep better structure layout. Experiments on various benchmarks demonstrate the excellent performance and fast inference speed of proposed approaches. Code is available at https://github.com/ltkong218/IFRNet.