# Towards Real-World Video Deblurring by Exploring Blur Formation Process

arXiv: https://arxiv.org/abs/2208.13184 · HTML: https://mlsanity.com/paper/2208.13184/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 28 Aug 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring](https://mlsanity.com/paper/2004.05343/) (2020)
- [MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement](https://mlsanity.com/paper/1810.08768/) (2018)
- [Spatio-Temporal Filter Adaptive Network for Video Deblurring](https://mlsanity.com/paper/1904.12257/) (2019)
- [Cascaded Deep Video Deblurring Using Temporal Sharpness Prior](https://mlsanity.com/paper/2004.02501/) (2020)
- [Deep Semantic Face Deblurring](https://mlsanity.com/paper/1803.03345/) (2018)
- [Adversarial Spatio-Temporal Learning for Video Deblurring](https://mlsanity.com/paper/1804.00533/) (2018)
- [ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring](https://mlsanity.com/paper/2103.04260/) (2021)
- [Online Video Deblurring via Dynamic Temporal Blending Network](https://mlsanity.com/paper/1704.03285/) (2017)
- [Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes](https://mlsanity.com/paper/2108.09982/) (2021)
- [Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes](https://mlsanity.com/paper/2104.01601/) (2021)
- [Deblurring using Analysis-Synthesis Networks Pair](https://mlsanity.com/paper/2004.02956/) (2020)
- [Raw Image Deblurring](https://mlsanity.com/paper/2012.04264/) (2020)
- [Animation from Blur: Multi-modal Blur Decomposition with Motion Guidance](https://mlsanity.com/paper/2207.10123/) (2022)
- [Efficient Video Deblurring Guided by Motion Magnitude](https://mlsanity.com/paper/2207.13374/) (2022)
- [VDTR: Video Deblurring with Transformer](https://mlsanity.com/paper/2204.08023/) (2022)

## Built on by (most cited first, 1 of 1)

- [Motion-Aware Animatable Gaussian Avatars Deblurring](https://mlsanity.com/paper/2411.16758/) (2024)

## Abstract (arXiv)

This paper aims at exploring how to synthesize close-to-real blurs that existing video deblurring models trained on them can generalize well to real-world blurry videos. In recent years, deep learning-based approaches have achieved promising success on video deblurring task. However, the models trained on existing synthetic datasets still suffer from generalization problems over real-world blurry scenarios with undesired artifacts. The factors accounting for the failure remain unknown. Therefore, we revisit the classical blur synthesis pipeline and figure out the possible reasons, including shooting parameters, blur formation space, and image signal processor~(ISP). To analyze the effects of these potential factors, we first collect an ultra-high frame-rate (940 FPS) RAW video dataset as the data basis to synthesize various kinds of blurs. Then we propose a novel realistic blur synthesis pipeline termed as RAW-Blur by leveraging blur formation cues. Through numerous experiments, we demonstrate that synthesizing blurs in the RAW space and adopting the same ISP as the real-world testing data can effectively eliminate the negative effects of synthetic data. Furthermore, the shooting parameters of the synthesized blurry video, e.g., exposure time and frame-rate play significant roles in improving the performance of deblurring models. Impressively, the models trained on the blurry data synthesized by the proposed RAW-Blur pipeline can obtain more than 5dB PSNR gain against those trained on the existing synthetic blur datasets. We believe the novel realistic synthesis pipeline and the corresponding RAW video dataset can help the community to easily construct customized blur datasets to improve real-world video deblurring performance largely, instead of laboriously collecting real data pairs.