# EA-Net: Edge-Aware Network for Flow-based Video Frame Interpolation

arXiv: https://arxiv.org/abs/2105.07673 · HTML: https://mlsanity.com/paper/2105.07673/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 62 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [XVFI: eXtreme Video Frame Interpolation](https://mlsanity.com/paper/2103.16206/) (2021)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution](https://mlsanity.com/paper/2002.11616/) (2020)
- [PhaseNet for Video Frame Interpolation](https://mlsanity.com/paper/1804.00884/) (2018)
- [Blurry Video Frame Interpolation](https://mlsanity.com/paper/2002.12259/) (2020)
- [SDCNet: Video Prediction Using Spatially-Displaced Convolution](https://mlsanity.com/paper/1811.00684/) (2018)
- [Spatiotemporal Co-attention Recurrent Neural Networks for Human-Skeleton Motion Prediction](https://mlsanity.com/paper/1909.13245/) (2019)
- [Reconstructive Sequence-Graph Network for Video Summarization](https://mlsanity.com/paper/2105.04066/) (2021)
- [Scene-Adaptive Video Frame Interpolation via Meta-Learning](https://mlsanity.com/paper/2004.00779/) (2020)
- [Temporally Identity-Aware SSD with Attentional LSTM](https://mlsanity.com/paper/1803.00197/) (2018)

## Built on by (most cited first, 6 of 6)

- [Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation](https://mlsanity.com/paper/2203.10291/) (2022)
- [STDAN: Deformable Attention Network for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2203.06841/) (2022)
- [STEdge: Self-training Edge Detection with Multi-layer Teaching and Regularization](https://mlsanity.com/paper/2201.05121/) (2022)
- [Progressive Motion Context Refine Network for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2211.06024/) (2022)
- [Motion-Aware Video Frame Interpolation](https://mlsanity.com/paper/2402.02892/) (2024)
- [Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning](https://mlsanity.com/paper/2411.02983/) (2024)

## Abstract (arXiv)

Video frame interpolation can up-convert the frame rate and enhance the video quality. In recent years, although the interpolation performance has achieved great success, image blur usually occurs at the object boundaries owing to the large motion. It has been a long-standing problem, and has not been addressed yet. In this paper, we propose to reduce the image blur and get the clear shape of objects by preserving the edges in the interpolated frames. To this end, the proposed Edge-Aware Network (EA-Net) integrates the edge information into the frame interpolation task. It follows an end-to-end architecture and can be separated into two stages, i.e., edge-guided flow estimation and edge-protected frame synthesis. Specifically, in the flow estimation stage, three edge-aware mechanisms are developed to emphasize the frame edges in estimating flow maps, so that the edge-maps are taken as the auxiliary information to provide more guidance to boost the flow accuracy. In the frame synthesis stage, the flow refinement module is designed to refine the flow map, and the attention module is carried out to adaptively focus on the bidirectional flow maps when synthesizing the intermediate frames. Furthermore, the frame and edge discriminators are adopted to conduct the adversarial training strategy, so as to enhance the reality and clarity of synthesized frames. Experiments on three benchmarks, including Vimeo90k, UCF101 for single-frame interpolation and Adobe240-fps for multi-frame interpolation, have demonstrated the superiority of the proposed EA-Net for the video frame interpolation task.