# Deep Dynamic Scene Deblurring from Optical Flow

arXiv: https://arxiv.org/abs/2301.07329 · HTML: https://mlsanity.com/paper/2301.07329/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 18 Jan 2023.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal](https://mlsanity.com/paper/1503.00593/) (2015)
- [Spatio-Temporal Filter Adaptive Network for Video Deblurring](https://mlsanity.com/paper/1904.12257/) (2019)
- [Cascaded Deep Video Deblurring Using Temporal Sharpness Prior](https://mlsanity.com/paper/2004.02501/) (2020)
- [From Motion Blur to Motion Flow: a Deep Learning Solution for Removing Heterogeneous Motion Blur](https://mlsanity.com/paper/1612.02583/) (2016)
- [Online Video Deblurring via Dynamic Temporal Blending Network](https://mlsanity.com/paper/1704.03285/) (2017)
- [Generalized Video Deblurring for Dynamic Scenes](https://mlsanity.com/paper/1507.02438/) (2015)
- [Reblur2Deblur: Deblurring Videos via Self-Supervised Learning](https://mlsanity.com/paper/1801.05117/) (2018)
- [Motion Deblurring in the Wild](https://mlsanity.com/paper/1701.01486/) (2017)
- [Stereo Video Deblurring](https://mlsanity.com/paper/1607.08421/) (2016)
- [Simultaneous Stereo Video Deblurring and Scene Flow Estimation](https://mlsanity.com/paper/1704.03273/) (2017)
- [A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection](https://mlsanity.com/paper/2012.10162/) (2020)
- [Occlusion-Aware Video Deblurring with a New Layered Blur Model](https://mlsanity.com/paper/1611.09572/) (2016)

## Built on by (most cited first, 5 of 5)

- [OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation](https://mlsanity.com/paper/2403.18092/) (2024)
- [Adaptive Window Pruning for Efficient Local Motion Deblurring](https://mlsanity.com/paper/2306.14268/) (2023)
- [Deep learning in motion deblurring: current status, benchmarks and future prospects](https://mlsanity.com/paper/2401.05055/) (2024)
- [DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis](https://mlsanity.com/paper/2602.23022/) (2026)
- [Leveraging the Powerful Attention of a Pre-trained Diffusion Model for Exemplar-based Image Colorization](https://mlsanity.com/paper/2505.15812/) (2025)

## Abstract (arXiv)

Deblurring can not only provide visually more pleasant pictures and make photography more convenient, but also can improve the performance of objection detection as well as tracking. However, removing dynamic scene blur from images is a non-trivial task as it is difficult to model the non-uniform blur mathematically. Several methods first use single or multiple images to estimate optical flow (which is treated as an approximation of blur kernels) and then adopt non-blind deblurring algorithms to reconstruct the sharp images. However, these methods cannot be trained in an end-to-end manner and are usually computationally expensive. In this paper, we explore optical flow to remove dynamic scene blur by using the multi-scale spatially variant recurrent neural network (RNN). We utilize FlowNets to estimate optical flow from two consecutive images in different scales. The estimated optical flow provides the RNN weights in different scales so that the weights can better help RNNs to remove blur in the feature spaces. Finally, we develop a convolutional neural network (CNN) to restore the sharp images from the deblurred features. Both quantitative and qualitative evaluations on the benchmark datasets demonstrate that the proposed method performs favorably against state-of-the-art algorithms in terms of accuracy, speed, and model size.