# Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework

arXiv: https://arxiv.org/abs/1803.10433 · HTML: https://mlsanity.com/paper/1803.10433/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2018.

## Built on by (most cited first, 30 of 53)

- [Restoring Images in Adverse Weather Conditions via Histogram Transformer](https://mlsanity.com/paper/2407.10172/) (2024)
- [RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining](https://mlsanity.com/paper/2407.21773/) (2024)
- [Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation](https://mlsanity.com/paper/2103.13660/) (2021)
- [Semi-Supervised Video Deraining with Dynamical Rain Generator](https://mlsanity.com/paper/2103.07939/) (2021)
- [Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework](https://mlsanity.com/paper/2103.12318/) (2021)
- [Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal](https://mlsanity.com/paper/2103.07051/) (2021)
- [Neural Compression-Based Feature Learning for Video Restoration](https://mlsanity.com/paper/2203.09208/) (2022)
- [Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagation](https://mlsanity.com/paper/2309.13700/) (2023)
- [An Annotation-free Restoration Network for Cataractous Fundus Images](https://mlsanity.com/paper/2203.07737/) (2022)
- [Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Prior](https://mlsanity.com/paper/2105.03830/) (2021)
- [Neural Image Representations for Multi-Image Fusion and Layer Separation](https://mlsanity.com/paper/2108.01199/) (2021)
- [Structure-consistent Restoration Network for Cataract Fundus Image Enhancement](https://mlsanity.com/paper/2206.04684/) (2022)
- [GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration](https://mlsanity.com/paper/2411.17687/) (2024)
- [RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining](https://mlsanity.com/paper/2107.06808/) (2021)
- [Unsupervised Image Deraining: Optimization Model Driven Deep CNN](https://mlsanity.com/paper/2203.13699/) (2022)
- [Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal](https://mlsanity.com/paper/2403.07684/) (2024)
- [A Generic Fundus Image Enhancement Network Boosted by Frequency Self-supervised Representation Learning](https://mlsanity.com/paper/2309.00885/) (2023)
- [Let's See Clearly: Contaminant Artifact Removal for Moving Cameras](https://mlsanity.com/paper/2104.08852/) (2021)
- [Triple-level Model Inferred Collaborative Network Architecture for Video Deraining](https://mlsanity.com/paper/2111.04459/) (2021)
- [RainyScape: Unsupervised Rainy Scene Reconstruction using Decoupled Neural Rendering](https://mlsanity.com/paper/2404.11401/) (2024)
- [Driving-Video Dehazing with Non-Aligned Regularization for Safety Assistance](https://mlsanity.com/paper/2405.09996/) (2024)
- [Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining](https://mlsanity.com/paper/2505.16811/) (2025)
- [SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-Trained Siamese Transformers](https://mlsanity.com/paper/2112.09426/) (2021)
- [EGVD: Event-Guided Video Deraining](https://mlsanity.com/paper/2309.17239/) (2023)
- [MODEM: A Morton-Order Degradation Estimation Mechanism for Adverse Weather Image Recovery](https://mlsanity.com/paper/2505.17581/) (2025)
- [An End-to-End Cascaded Image Deraining and Object Detection Neural Network](https://mlsanity.com/paper/2202.11279/) (2022)
- [Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration](https://mlsanity.com/paper/2309.01627/) (2023)
- [PRE-Mamba: A 4D State Space Model for Ultra-High-Frequent Event Camera Deraining](https://mlsanity.com/paper/2505.05307/) (2025)
- [DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining](https://mlsanity.com/paper/2509.21719/) (2025)
- [ASF-Net: Robust Video Deraining via Temporal Alignment and Online Adaptive Learning](https://mlsanity.com/paper/2309.00956/) (2023)

## Abstract (arXiv)

Rain removal is important for improving the robustness of outdoor vision based systems. Current rain removal methods show limitations either for complex dynamic scenes shot from fast moving cameras, or under torrential rain fall with opaque occlusions. We propose a novel derain algorithm, which applies superpixel (SP) segmentation to decompose the scene into depth consistent units. Alignment of scene contents are done at the SP level, which proves to be robust towards rain occlusion and fast camera motion. Two alignment output tensors, i.e., optimal temporal match tensor and sorted spatial-temporal match tensor, provide informative clues for rain streak location and occluded background contents to generate an intermediate derain output. These tensors will be subsequently prepared as input features for a convolutional neural network to restore high frequency details to the intermediate output for compensation of mis-alignment blur. Extensive evaluations show that up to 5 dB reconstruction PSNR advantage is achieved over state-of-the-art methods. Visual inspection shows that much cleaner rain removal is achieved especially for highly dynamic scenes with heavy and opaque rainfall from a fast moving camera.