# SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams

arXiv: https://arxiv.org/abs/2503.20315 · HTML: https://mlsanity.com/paper/2503.20315/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 55 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [DETRs with Hybrid Matching](https://mlsanity.com/paper/2207.13080/) (2022)
- [GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions](https://mlsanity.com/paper/2305.17863/) (2023)
- [Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework](https://mlsanity.com/paper/1803.10433/) (2018)
- [Masked Spiking Transformer](https://mlsanity.com/paper/2210.01208/) (2022)
- [Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks](https://mlsanity.com/paper/2312.06372/) (2023)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)
- [Semi-Supervised Video Deraining with Dynamical Rain Generator](https://mlsanity.com/paper/2103.07939/) (2021)
- [Mitigating Motion Blur in Neural Radiance Fields with Events and Frames](https://mlsanity.com/paper/2403.19780/) (2024)
- [Enhanced Spatio-Temporal Interaction Learning for Video Deraining: A Faster and Better Framework](https://mlsanity.com/paper/2103.12318/) (2021)
- [Line-based 6-DoF Object Pose Estimation and Tracking With an Event Camera](https://mlsanity.com/paper/2408.03225/) (2024)
- [Unifying Activation- and Timing-based Learning Rules for Spiking Neural Networks](https://mlsanity.com/paper/2006.02642/) (2020)
- [Joint Self-Attention and Scale-Aggregation for Self-Calibrated Deraining Network](https://mlsanity.com/paper/2008.02763/) (2020)
- [1000x Faster Camera and Machine Vision with Ordinary Devices](https://mlsanity.com/paper/2201.09302/) (2022)
- [Spike Camera and Its Coding Methods](https://mlsanity.com/paper/2104.04669/) (2021)
- [Stereo Event-based, 6-DOF Pose Tracking for Uncooperative Spacecraft](https://mlsanity.com/paper/2503.12732/) (2025)
- [Learning A Spiking Neural Network for Efficient Image Deraining](https://mlsanity.com/paper/2405.06277/) (2024)
- [An Efficient Coding Method for Spike Camera using Inter-Spike Intervals](https://mlsanity.com/paper/1912.09669/) (2019)
- [Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks](https://mlsanity.com/paper/2409.12507/) (2024)
- [SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams](https://mlsanity.com/paper/2403.09486/) (2024)

## Built on by (most cited first, 1 of 1)

- [RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics](https://mlsanity.com/paper/2607.13802/) (2026)

## Abstract (arXiv)

Restoring clear frames from rainy videos presents a significant challenge due to the rapid motion of rain streaks. Traditional frame-based visual sensors, which capture scene content synchronously, struggle to capture the fast-moving details of rain accurately. In recent years, neuromorphic sensors have introduced a new paradigm for dynamic scene perception, offering microsecond temporal resolution and high dynamic range. However, existing multimodal methods that fuse event streams with RGB images face difficulties in handling the complex spatiotemporal interference of raindrops in real scenes, primarily due to hardware synchronization errors and computational redundancy. In this paper, we propose a Color Spike Stream Deraining Network (SpikeDerain), capable of reconstructing spike streams of dynamic scenes and accurately removing rain streaks. To address the challenges of data scarcity in real continuous rainfall scenes, we design a physically interpretable rain streak synthesis model that generates parameterized continuous rain patterns based on arbitrary background images. Experimental results demonstrate that the network, trained with this synthetic data, remains highly robust even under extreme rainfall conditions. These findings highlight the effectiveness and robustness of our method across varying rainfall levels and datasets, setting new standards for video deraining tasks. The code will be released soon.