# ASF-Net: Robust Video Deraining via Temporal Alignment and Online Adaptive Learning

arXiv: https://arxiv.org/abs/2309.00956 · HTML: https://mlsanity.com/paper/2309.00956/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 2 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement](https://mlsanity.com/paper/2012.05609/) (2020)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Real-world Underwater Enhancement: Challenges, Benchmarks, and Solutions](https://mlsanity.com/paper/1901.05320/) (2019)
- [Single Image Deraining: From Model-Based to Data-Driven and Beyond](https://mlsanity.com/paper/1912.07150/) (2019)
- [Syn2Real Transfer Learning for Image Deraining using Gaussian Processes](https://mlsanity.com/paper/2006.05580/) (2020)
- [Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework](https://mlsanity.com/paper/1803.10433/) (2018)
- [Essential Tensor Learning for Multi-view Spectral Clustering](https://mlsanity.com/paper/1807.03602/) (2018)
- [Semi-Supervised Video Deraining with Dynamical Rain Generator](https://mlsanity.com/paper/2103.07939/) (2021)
- [Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework](https://mlsanity.com/paper/2103.12318/) (2021)
- [FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors](https://mlsanity.com/paper/1803.07487/) (2018)
- [Bilevel Fast Scene Adaptation for Low-Light Image Enhancement](https://mlsanity.com/paper/2306.01343/) (2023)
- [Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond](https://mlsanity.com/paper/2004.14557/) (2020)
- [Automated Learning for Deformable Medical Image Registration by Jointly Optimizing Network Architectures and Objective Functions](https://mlsanity.com/paper/2203.06810/) (2022)
- [Removing Rain in Videos: A Large-scale Database and A Two-stream ConvLSTM Approach](https://mlsanity.com/paper/1906.02526/) (2019)

## Built on by (most cited first, 3 of 3)

- [UENR-600K: A Large-Scale Physically Grounded Dataset for Nighttime Video Deraining](https://mlsanity.com/paper/2604.04402/) (2026)
- [S3VD: Semantic-Guidance Spatio-Temporal Scanning for Video Deraining](https://mlsanity.com/paper/2609.21322/) (2026)
- [FluidRain: Incompressible Rain Flow as an Attention Bias for Loop-in-Loop Video Deraining](https://mlsanity.com/paper/2609.29006/) (2026)

## Abstract (arXiv)

In recent times, learning-based methods for video deraining have demonstrated commendable results. However, there are two critical challenges that these methods are yet to address: exploiting temporal correlations among adjacent frames and ensuring adaptability to unknown real-world scenarios. To overcome these challenges, we explore video deraining from a paradigm design perspective to learning strategy construction. Specifically, we propose a new computational paradigm, Alignment-Shift-Fusion Network (ASF-Net), which incorporates a temporal shift module. This module is novel to this field and provides deeper exploration of temporal information by facilitating the exchange of channel-level information within the feature space. To fully discharge the model's characterization capability, we further construct a LArge-scale RAiny video dataset (LARA) which also supports the development of this community. On the basis of the newly-constructed dataset, we explore the parameters learning process by developing an innovative re-degraded learning strategy. This strategy bridges the gap between synthetic and real-world scenes, resulting in stronger scene adaptability. Our proposed approach exhibits superior performance in three benchmarks and compelling visual quality in real-world scenarios, underscoring its efficacy. The code is available at https://github.com/vis-opt-group/ASF-Net.