# MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement

arXiv: https://arxiv.org/abs/1810.08768 · HTML: https://mlsanity.com/paper/1810.08768/

## Key facts

- Cited by 112 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2018.

## Built on by (most cited first, 30 of 112)

- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [Asymmetric Bilateral Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2108.06815/) (2021)
- [Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2303.00440/) (2023)
- [LDMVFI: Video Frame Interpolation with Latent Diffusion Models](https://mlsanity.com/paper/2303.09508/) (2023)
- [Rethinking Alignment in Video Super-Resolution Transformers](https://mlsanity.com/paper/2207.08494/) (2022)
- [Real-World Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/2103.02368/) (2021)
- [Deep Animation Video Interpolation in the Wild](https://mlsanity.com/paper/2104.02495/) (2021)
- [CDFI: Compression-Driven Network Design for Frame Interpolation](https://mlsanity.com/paper/2103.10559/) (2021)
- [Temporal Modulation Network for Controllable Space-Time Video Super-Resolution](https://mlsanity.com/paper/2104.10642/) (2021)
- [RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2203.14186/) (2022)
- [Many-to-many Splatting for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2204.03513/) (2022)
- [A Unified Pyramid Recurrent Network for Video Frame Interpolation](https://mlsanity.com/paper/2211.03456/) (2022)
- [TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolation](https://mlsanity.com/paper/2203.13859/) (2022)
- [ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation](https://mlsanity.com/paper/2111.15483/) (2021)
- [Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution](https://mlsanity.com/paper/2303.13767/) (2023)
- [Optimizing Video Prediction via Video Frame Interpolation](https://mlsanity.com/paper/2206.13454/) (2022)
- [Event-Based Video Frame Interpolation With Cross-Modal Asymmetric Bidirectional Motion Fields](https://mlsanity.com/paper/2502.13716/) (2025)
- [Video Frame Interpolation without Temporal Priors](https://mlsanity.com/paper/2112.01161/) (2021)
- [Framer: Interactive Frame Interpolation](https://mlsanity.com/paper/2410.18978/) (2024)
- [Progressively Complementary Network for Fisheye Image Rectification Using Appearance Flow](https://mlsanity.com/paper/2103.16026/) (2021)
- [BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolation](https://mlsanity.com/paper/2304.02225/) (2023)
- [Perception-Oriented Video Frame Interpolation via Asymmetric Blending](https://mlsanity.com/paper/2404.06692/) (2024)
- [Drag-A-Video: Non-rigid Video Editing with Point-based Interaction](https://mlsanity.com/paper/2312.02936/) (2023)
- [Improving the Perceptual Quality of 2D Animation Interpolation](https://mlsanity.com/paper/2111.12792/) (2021)
- [Learning for Unconstrained Space-Time Video Super-Resolution](https://mlsanity.com/paper/2102.13011/) (2021)
- [DeMFI: Deep Joint Deblurring and Multi-Frame Interpolation with Flow-Guided Attentive Correlation and Recursive Boosting](https://mlsanity.com/paper/2111.09985/) (2021)
- [Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation](https://mlsanity.com/paper/2203.10291/) (2022)
- [Enhanced Bi-directional Motion Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2206.08572/) (2022)
- [NTIRE 2021 Challenge on Video Super-Resolution](https://mlsanity.com/paper/2104.14852/) (2021)

## Abstract (arXiv)

Motion estimation (ME) and motion compensation (MC) have been widely used for classical video frame interpolation systems over the past decades. Recently, a number of data-driven frame interpolation methods based on convolutional neural networks have been proposed. However, existing learning based methods typically estimate either flow or compensation kernels, thereby limiting performance on both computational efficiency and interpolation accuracy. In this work, we propose a motion estimation and compensation driven neural network for video frame interpolation. A novel adaptive warping layer is developed to integrate both optical flow and interpolation kernels to synthesize target frame pixels. This layer is fully differentiable such that both the flow and kernel estimation networks can be optimized jointly. The proposed model benefits from the advantages of motion estimation and compensation methods without using hand-crafted features. Compared to existing methods, our approach is computationally efficient and able to generate more visually appealing results. Furthermore, the proposed MEMC-Net can be seamlessly adapted to several video enhancement tasks, e.g., super-resolution, denoising, and deblocking. Extensive quantitative and qualitative evaluations demonstrate that the proposed method performs favorably against the state-of-the-art video frame interpolation and enhancement algorithms on a wide range of datasets.