# Deep Recurrent Neural Network with Multi-scale Bi-directional Propagation for Video Deblurring

arXiv: https://arxiv.org/abs/2112.05150 · HTML: https://mlsanity.com/paper/2112.05150/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Rethinking Coarse-to-Fine Approach in Single Image Deblurring](https://mlsanity.com/paper/2108.05054/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Deep Stacked Hierarchical Multi-patch Network for Image Deblurring](https://mlsanity.com/paper/1904.03468/) (2019)
- [Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring](https://mlsanity.com/paper/2004.05343/) (2020)
- [Spatio-Temporal Filter Adaptive Network for Video Deblurring](https://mlsanity.com/paper/1904.12257/) (2019)
- [Cascaded Deep Video Deblurring Using Temporal Sharpness Prior](https://mlsanity.com/paper/2004.02501/) (2020)
- [From Motion Blur to Motion Flow: a Deep Learning Solution for Removing Heterogeneous Motion Blur](https://mlsanity.com/paper/1612.02583/) (2016)
- [Adversarial Spatio-Temporal Learning for Video Deblurring](https://mlsanity.com/paper/1804.00533/) (2018)
- [Online Video Deblurring via Dynamic Temporal Blending Network](https://mlsanity.com/paper/1704.03285/) (2017)
- [NTIRE 2021 Challenge on Image Deblurring](https://mlsanity.com/paper/2104.14854/) (2021)
- [Generalized Video Deblurring for Dynamic Scenes](https://mlsanity.com/paper/1507.02438/) (2015)
- [Learning to Synthesize Motion Blur](https://mlsanity.com/paper/1811.11745/) (2018)
- [DAVANet: Stereo Deblurring with View Aggregation](https://mlsanity.com/paper/1904.05065/) (2019)
- [Gated Fusion Network for Joint Image Deblurring and Super-Resolution](https://mlsanity.com/paper/1807.10806/) (2018)
- [NTIRE 2020 Challenge on Image and Video Deblurring](https://mlsanity.com/paper/2005.01244/) (2020)
- [Deep Video Deblurring: The Devil is in the Details](https://mlsanity.com/paper/1909.12196/) (2019)

## Built on by (most cited first, 25 of 25)

- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift](https://mlsanity.com/paper/2206.10810/) (2022)
- [Spatio-Temporal Turbulence Mitigation: A Translational Perspective](https://mlsanity.com/paper/2401.04244/) (2024)
- [Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring](https://mlsanity.com/paper/2406.07551/) (2024)
- [VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision](https://mlsanity.com/paper/2306.10681/) (2023)
- [Learning Truncated Causal History Model for Video Restoration](https://mlsanity.com/paper/2410.03936/) (2024)
- [CMTA: Cross-Modal Temporal Alignment for Event-guided Video Deblurring](https://mlsanity.com/paper/2408.14930/) (2024)
- [Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation](https://mlsanity.com/paper/2504.02697/) (2025)
- [AIM 2022 Challenge on Instagram Filter Removal: Methods and Results](https://mlsanity.com/paper/2210.08997/) (2022)
- [Event-Enhanced Blurry Video Super-Resolution](https://mlsanity.com/paper/2504.13042/) (2025)
- [Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration](https://mlsanity.com/paper/2309.01627/) (2023)
- [Deep learning in motion deblurring: current status, benchmarks and future prospects](https://mlsanity.com/paper/2401.05055/) (2024)
- [Learning to Deblur Polarized Images](https://mlsanity.com/paper/2402.18134/) (2024)
- [MS-LSTM: Exploring Spatiotemporal Multiscale Representations in Video Prediction Domain](https://mlsanity.com/paper/2304.07724/) (2023)
- [DAVIDE: Depth-Aware Video Deblurring](https://mlsanity.com/paper/2409.01274/) (2024)
- [Grounding Degradations in Natural Language for All-In-One Video Restoration](https://mlsanity.com/paper/2507.14851/) (2025)
- [EGTM: Event-guided Efficient Turbulence Mitigation](https://mlsanity.com/paper/2509.03808/) (2025)
- [CompEvent: Complex-valued Event-RGB Fusion for Low-light Video Enhancement and Deblurring](https://mlsanity.com/paper/2511.14469/) (2025)
- [ViStripformer: A Token-Efficient Transformer for Versatile Video Restoration](https://mlsanity.com/paper/2312.14502/) (2023)
- [SelfHVD: Self-Supervised Handheld Video Deblurring](https://mlsanity.com/paper/2508.08605/) (2025)
- [Large-FOV RGBD Imaging via Structured PSF Coding in a Bio-inspired Monocentric System](https://mlsanity.com/paper/2510.25314/) (2025)
- [FMA-Net++: Motion- and Exposure-Aware Joint Video Super-Resolution and Deblurring](https://mlsanity.com/paper/2512.04390/) (2025)
- [Spatio-Temporal Difference Guided Motion Deblurring with the Complementary Vision Sensor](https://mlsanity.com/paper/2604.10554/) (2026)
- [GS-RealBlur: A Flexible Data Acquisition Framework for Real-World Image Deblurring](https://mlsanity.com/paper/2607.15401/) (2026)
- [RealVDeblur: One-Step Diffusion for Generalizable Real-World Video Deblurring](https://mlsanity.com/paper/2607.20628/) (2026)

## Abstract (arXiv)

The success of the state-of-the-art video deblurring methods stems mainly from implicit or explicit estimation of alignment among the adjacent frames for latent video restoration. However, due to the influence of the blur effect, estimating the alignment information from the blurry adjacent frames is not a trivial task. Inaccurate estimations will interfere the following frame restoration. Instead of estimating alignment information, we propose a simple and effective deep Recurrent Neural Network with Multi-scale Bi-directional Propagation (RNN-MBP) to effectively propagate and gather the information from unaligned neighboring frames for better video deblurring. Specifically, we build a Multi-scale Bi-directional Propagation~(MBP) module with two U-Net RNN cells which can directly exploit the inter-frame information from unaligned neighboring hidden states by integrating them in different scales. Moreover, to better evaluate the proposed algorithm and existing state-of-the-art methods on real-world blurry scenes, we also create a Real-World Blurry Video Dataset (RBVD) by a well-designed Digital Video Acquisition System (DVAS) and use it as the training and evaluation dataset. Extensive experimental results demonstrate that the proposed RBVD dataset effectively improves the performance of existing algorithms on real-world blurry videos, and the proposed algorithm performs favorably against the state-of-the-art methods on three typical benchmarks. The code is available at https://github.com/XJTU-CVLAB-LOWLEVEL/RNN-MBP.