# RainMamba: Enhanced Locality Learning with State Space Models for Video Deraining

arXiv: https://arxiv.org/abs/2407.21773 · HTML: https://mlsanity.com/paper/2407.21773/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 40 of its 90 references that are in the map.
- First posted on arXiv 31 Jul 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Vision Mamba backbones](https://mlsanity.com/ssm/vision-mamba-backbones/)

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [VideoMamba: State Space Model for Efficient Video Understanding](https://mlsanity.com/paper/2403.06977/) (2024)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [LocalMamba: Visual State Space Model with Windowed Selective Scan](https://mlsanity.com/paper/2403.09338/) (2024)
- [Contrastive Learning for Compact Single Image Dehazing](https://mlsanity.com/paper/2104.09367/) (2021)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Recurrent Video Restoration Transformer with Guided Deformable Attention](https://mlsanity.com/paper/2206.02146/) (2022)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)
- [VmambaIR: Visual State Space Model for Image Restoration](https://mlsanity.com/paper/2403.11423/) (2024)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [Unpaired Deep Image Deraining Using Dual Contrastive Learning](https://mlsanity.com/paper/2109.02973/) (2021)
- [Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework](https://mlsanity.com/paper/1803.10433/) (2018)
- [Semi-Supervised Video Deraining with Dynamical Rain Generator](https://mlsanity.com/paper/2103.07939/) (2021)
- [Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework](https://mlsanity.com/paper/2103.12318/) (2021)
- [AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors](https://mlsanity.com/paper/2403.04697/) (2024)
- [DRT: A Lightweight Single Image Deraining Recursive Transformer](https://mlsanity.com/paper/2204.11385/) (2022)

## Built on by (most cited first, 30 of 56)

- [MambaIRv2: Attentive State Space Restoration](https://mlsanity.com/paper/2411.15269/) (2024)
- [MedGround-R1: Advancing Medical Image Grounding via Spatial-Semantic Rewarded Group Relative Policy Optimization](https://mlsanity.com/paper/2507.02994/) (2025)
- [NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results](https://mlsanity.com/paper/2504.12711/) (2025)
- [MambaCSR: Dual-Interleaved Scanning for Compressed Image Super-Resolution With SSMs](https://mlsanity.com/paper/2408.11758/) (2024)
- [NTIRE 2026 The Second Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results](https://mlsanity.com/paper/2604.10634/) (2026)
- [NTIRE 2025 Challenge on Image Super-Resolution (x4): Methods and Results](https://mlsanity.com/paper/2504.14582/) (2025)
- [MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration](https://mlsanity.com/paper/2510.08508/) (2025)
- [Controllable Weather Synthesis and Removal with Video Diffusion Models](https://mlsanity.com/paper/2505.00704/) (2025)
- [Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation](https://mlsanity.com/paper/2504.02697/) (2025)
- [Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining](https://mlsanity.com/paper/2505.16811/) (2025)
- [CenterMamba-SAM: Center-Prioritized Scanning and Temporal Prototypes for Brain Lesion Segmentation](https://mlsanity.com/paper/2511.01243/) (2025)
- [Language-Driven Interactive Shadow Detection](https://mlsanity.com/paper/2408.08543/) (2024)
- [Timeline and Boundary Guided Diffusion Network for Video Shadow Detection](https://mlsanity.com/paper/2408.11785/) (2024)
- [Non-Invasive to Invasive: Enhancing FFA Synthesis from CFP with a Benchmark Dataset and a Novel Network](https://mlsanity.com/paper/2410.14965/) (2024)
- [EventMamba: Enhancing Spatio-Temporal Locality with State Space Models for Event-Based Video Reconstruction](https://mlsanity.com/paper/2503.19721/) (2025)
- [Content-Aware Mamba for Learned Image Compression](https://mlsanity.com/paper/2508.02192/) (2025)
- [DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining](https://mlsanity.com/paper/2509.21719/) (2025)
- [Alignment-free Raw Video Demoireing](https://mlsanity.com/paper/2408.10679/) (2024)
- [RAWMamba: Unified sRGB-to-RAW De-rendering With State Space Model](https://mlsanity.com/paper/2411.11717/) (2024)
- [OSMamba: Omnidirectional Spectral Mamba with Dual-Domain Prior Generator for Exposure Correction](https://mlsanity.com/paper/2411.15255/) (2024)
- [DPMambaIR: All-in-One Image Restoration via Degradation-Aware Prompt State Space Model](https://mlsanity.com/paper/2504.17732/) (2025)
- [Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarks](https://mlsanity.com/paper/2511.05598/) (2025)
- [Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models](https://mlsanity.com/paper/2503.11224/) (2025)
- [MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment](https://mlsanity.com/paper/2504.16003/) (2025)
- [Zero-Shot Video Deraining with Video Diffusion Models](https://mlsanity.com/paper/2511.18537/) (2025)
- [Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarking](https://mlsanity.com/paper/2602.20680/) (2026)
- [Unpaired Image Deraining Using Reward-Guided Self-Reinforcement Strategy](https://mlsanity.com/paper/2605.00719/) (2026)
- [BadVim: Unveiling Backdoor Threats in Visual State Space Model](https://mlsanity.com/paper/2408.11679/) (2024)
- [ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal](https://mlsanity.com/paper/2411.03260/) (2024)
- [Geometrically Regularized Transfer Learning with On-Manifold and Off-Manifold Perturbation](https://mlsanity.com/paper/2505.15191/) (2025)

## Abstract (arXiv)

The outdoor vision systems are frequently contaminated by rain streaks and raindrops, which significantly degenerate the performance of visual tasks and multimedia applications. The nature of videos exhibits redundant temporal cues for rain removal with higher stability. Traditional video deraining methods heavily rely on optical flow estimation and kernel-based manners, which have a limited receptive field. Yet, transformer architectures, while enabling long-term dependencies, bring about a significant increase in computational complexity. Recently, the linear-complexity operator of the state space models (SSMs) has contrarily facilitated efficient long-term temporal modeling, which is crucial for rain streaks and raindrops removal in videos. Unexpectedly, its uni-dimensional sequential process on videos destroys the local correlations across the spatio-temporal dimension by distancing adjacent pixels. To address this, we present an improved SSMs-based video deraining network (RainMamba) with a novel Hilbert scanning mechanism to better capture sequence-level local information. We also introduce a difference-guided dynamic contrastive locality learning strategy to enhance the patch-level self-similarity learning ability of the proposed network. Extensive experiments on four synthesized video deraining datasets and real-world rainy videos demonstrate the effectiveness and efficiency of our network in the removal of rain streaks and raindrops. Our code and results are available at https://github.com/TonyHongtaoWu/RainMamba.