# Semi-Supervised Video Deraining with Dynamical Rain Generator

arXiv: https://arxiv.org/abs/2103.07939 · HTML: https://mlsanity.com/paper/2103.07939/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 68 references that are in the map.
- First posted on arXiv 14 Mar 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Clearing the Skies: A deep network architecture for single-image rain removal](https://mlsanity.com/paper/1609.02087/) (2016)
- [ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation](https://mlsanity.com/paper/1803.06815/) (2018)
- [A Model-driven Deep Neural Network for Single Image Rain Removal](https://mlsanity.com/paper/2005.01333/) (2020)
- [Uncertainty Guided Multi-Scale Residual Learning-using a Cycle Spinning CNN for Single Image De-Raining](https://mlsanity.com/paper/1906.11129/) (2019)
- [Lightweight Pyramid Networks for Image Deraining](https://mlsanity.com/paper/1805.06173/) (2018)
- [Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework](https://mlsanity.com/paper/1803.10433/) (2018)
- [Perceptual Adversarial Networks for Image-to-Image Transformation](https://mlsanity.com/paper/1706.09138/) (2017)
- [Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining](https://mlsanity.com/paper/1808.01491/) (2018)
- [FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors](https://mlsanity.com/paper/1803.07487/) (2018)
- [Learning Dynamic Generator Model by Alternating Back-Propagation Through Time](https://mlsanity.com/paper/1812.10587/) (2018)
- [Motion-Based Generator Model: Unsupervised Disentanglement of Appearance, Trackable and Intrackable Motions in Dynamic Patterns](https://mlsanity.com/paper/1911.11294/) (2019)
- [From Rain Generation to Rain Removal](https://mlsanity.com/paper/2008.03580/) (2020)

## Built on by (most cited first, 30 of 31)

- [RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining](https://mlsanity.com/paper/2407.21773/) (2024)
- [Not Just Streaks: Towards Ground Truth for Single Image Deraining](https://mlsanity.com/paper/2206.10779/) (2022)
- [Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework](https://mlsanity.com/paper/2103.12318/) (2021)
- [Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagation](https://mlsanity.com/paper/2309.13700/) (2023)
- [DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environments](https://mlsanity.com/paper/2408.11540/) (2024)
- [Learning Truncated Causal History Model for Video Restoration](https://mlsanity.com/paper/2410.03936/) (2024)
- [Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal](https://mlsanity.com/paper/2403.07684/) (2024)
- [HINT: High-quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attention](https://mlsanity.com/paper/2402.14185/) (2024)
- [Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization](https://mlsanity.com/paper/2410.07901/) (2024)
- [RainyScape: Unsupervised Rainy Scene Reconstruction using Decoupled Neural Rendering](https://mlsanity.com/paper/2404.11401/) (2024)
- [Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining](https://mlsanity.com/paper/2505.16811/) (2025)
- [Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity](https://mlsanity.com/paper/2211.00837/) (2022)
- [LMQFormer: A Laplace-Prior-Guided Mask Query Transformer for Lightweight Snow Removal](https://mlsanity.com/paper/2210.04787/) (2022)
- [EGVD: Event-Guided Video Deraining](https://mlsanity.com/paper/2309.17239/) (2023)
- [DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining](https://mlsanity.com/paper/2509.21719/) (2025)
- [Learning Physical-Spatio-Temporal Features for Video Shadow Removal](https://mlsanity.com/paper/2303.09370/) (2023)
- [ASF-Net: Robust Video Deraining via Temporal Alignment and Online Adaptive Learning](https://mlsanity.com/paper/2309.00956/) (2023)
- [TDM: Temporally-Consistent Diffusion Model for All-in-One Real-World Video Restoration](https://mlsanity.com/paper/2501.02269/) (2025)
- [VJT: A Video Transformer on Joint Tasks of Deblurring, Low-light Enhancement and Denoising](https://mlsanity.com/paper/2401.14754/) (2024)
- [Zero-Shot Video Deraining with Video Diffusion Models](https://mlsanity.com/paper/2511.18537/) (2025)
- [Removing Multiple Hybrid Adverse Weather in Video via a Unified Model](https://mlsanity.com/paper/2503.06200/) (2025)
- [SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams](https://mlsanity.com/paper/2503.20315/) (2025)
- [Event-Based De-Snowing for Autonomous Driving](https://mlsanity.com/paper/2507.20901/) (2025)
- [Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation](https://mlsanity.com/paper/2510.09228/) (2025)
- [UENR-600K: A Large-Scale Physically Grounded Dataset for Nighttime Video Deraining](https://mlsanity.com/paper/2604.04402/) (2026)
- [Pixel Ignores, Superpixel Sees: Adverse Weather Image Restoration via Semantic-Center SSM](https://mlsanity.com/paper/2608.01760/) (2026)
- [Uncertainty-Aware Cascaded Dilation Filtering for High-Efficiency Deraining](https://mlsanity.com/paper/2201.02366/) (2022)
- [Cross-Scenario Deraining Adaptation with Unpaired Data: Superpixel Structural Priors and Multi-Stage Pseudo-Rain Synthesis](https://mlsanity.com/paper/2603.21661/) (2026)
- [RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics](https://mlsanity.com/paper/2607.13802/) (2026)
- [S3VD: Semantic-Guidance Spatio-Temporal Scanning for Video Deraining](https://mlsanity.com/paper/2609.21322/) (2026)

## Abstract (arXiv)

While deep learning (DL)-based video deraining methods have achieved significant success recently, they still exist two major drawbacks. Firstly, most of them do not sufficiently model the characteristics of rain layers of rainy videos. In fact, the rain layers exhibit strong physical properties (e.g., direction, scale and thickness) in spatial dimension and natural continuities in temporal dimension, and thus can be generally modelled by the spatial-temporal process in statistics. Secondly, current DL-based methods seriously depend on the labeled synthetic training data, whose rain types are always deviated from those in unlabeled real data. Such gap between synthetic and real data sets leads to poor performance when applying them in real scenarios. Against these issues, this paper proposes a new semi-supervised video deraining method, in which a dynamic rain generator is employed to fit the rain layer, expecting to better depict its insightful characteristics. Specifically, such dynamic generator consists of one emission model and one transition model to simultaneously encode the spatially physical structure and temporally continuous changes of rain streaks, respectively, which both are parameterized as deep neural networks (DNNs). Further more, different prior formats are designed for the labeled synthetic and unlabeled real data, so as to fully exploit the common knowledge underlying them. Last but not least, we also design a Monte Carlo EM algorithm to solve this model. Extensive experiments are conducted to verify the superiorities of the proposed semi-supervised deraining model.