# Motion-blurred Video Interpolation and Extrapolation

arXiv: https://arxiv.org/abs/2103.02984 · HTML: https://mlsanity.com/paper/2103.02984/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 4 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation](https://mlsanity.com/paper/1907.10244/) (2019)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [Understanding Adversarial Examples from the Mutual Influence of Images and Perturbations](https://mlsanity.com/paper/2007.06189/) (2020)
- [Learning to Extract a Video Sequence from a Single Motion-Blurred Image](https://mlsanity.com/paper/1804.04065/) (2018)
- [Blurry Video Frame Interpolation](https://mlsanity.com/paper/2002.12259/) (2020)
- [Bringing Alive Blurred Moments](https://mlsanity.com/paper/1804.02913/) (2018)
- [Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy](https://mlsanity.com/paper/2010.13365/) (2020)
- [Generalized Video Deblurring for Dynamic Scenes](https://mlsanity.com/paper/1507.02438/) (2015)
- [CD-UAP: Class Discriminative Universal Adversarial Perturbation](https://mlsanity.com/paper/2010.03300/) (2020)
- [Universal Adversarial Perturbations Through the Lens of Deep Steganography: Towards A Fourier Perspective](https://mlsanity.com/paper/2102.06479/) (2021)
- [Data from Model: Extracting Data from Non-robust and Robust Models](https://mlsanity.com/paper/2007.06196/) (2020)

## Built on by (most cited first, 11 of 11)

- [Event-guided Deblurring of Unknown Exposure Time Videos](https://mlsanity.com/paper/2112.06988/) (2021)
- [Blur Interpolation Transformer for Real-World Motion from Blur](https://mlsanity.com/paper/2211.11423/) (2022)
- [Animation from Blur: Multi-modal Blur Decomposition with Motion Guidance](https://mlsanity.com/paper/2207.10123/) (2022)
- [Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time](https://mlsanity.com/paper/2303.15043/) (2023)
- [Restoration of Video Frames from a Single Blurred Image with Motion Understanding](https://mlsanity.com/paper/2104.09134/) (2021)
- [Towards Interpretable Video Super-Resolution via Alternating Optimization](https://mlsanity.com/paper/2207.10765/) (2022)
- [Long-term Video Frame Interpolation via Feature Propagation](https://mlsanity.com/paper/2203.15427/) (2022)
- [Video Reconstruction from a Single Motion Blurred Image using Learned Dynamic Phase Coding](https://mlsanity.com/paper/2112.14768/) (2021)
- [Meta Transferring for Deblurring](https://mlsanity.com/paper/2210.08036/) (2022)
- [Learning Spatio-Temporal Downsampling for Effective Video Upscaling](https://mlsanity.com/paper/2203.08140/) (2022)
- [NBD-GAP: Non-Blind Image Deblurring Without Clean Target Images](https://mlsanity.com/paper/2209.09498/) (2022)

## Abstract (arXiv)

Abrupt motion of camera or objects in a scene result in a blurry video, and therefore recovering high quality video requires two types of enhancements: visual enhancement and temporal upsampling. A broad range of research attempted to recover clean frames from blurred image sequences or temporally upsample frames by interpolation, yet there are very limited studies handling both problems jointly. In this work, we present a novel framework for deblurring, interpolating and extrapolating sharp frames from a motion-blurred video in an end-to-end manner. We design our framework by first learning the pixel-level motion that caused the blur from the given inputs via optical flow estimation and then predict multiple clean frames by warping the decoded features with the estimated flows. To ensure temporal coherence across predicted frames and address potential temporal ambiguity, we propose a simple, yet effective flow-based rule. The effectiveness and favorability of our approach are highlighted through extensive qualitative and quantitative evaluations on motion-blurred datasets from high speed videos.