# Towards Better De-raining Generalization via Rainy Characteristics Memorization and Replay

arXiv: https://arxiv.org/abs/2506.02477 · HTML: https://mlsanity.com/paper/2506.02477/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 69 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation](https://mlsanity.com/paper/2402.15852/) (2024)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks](https://mlsanity.com/paper/2412.06224/) (2024)
- [Clearing the Skies: A deep network architecture for single-image rain removal](https://mlsanity.com/paper/1609.02087/) (2016)
- [Single Image Deraining: From Model-Based to Data-Driven and Beyond](https://mlsanity.com/paper/1912.07150/) (2019)
- [Lightweight Pyramid Networks for Image Deraining](https://mlsanity.com/paper/1805.06173/) (2018)
- [Syn2Real Transfer Learning for Image Deraining using Gaussian Processes](https://mlsanity.com/paper/2006.05580/) (2020)
- [MaIR: A Locality- and Continuity-Preserving Mamba for Image Restoration](https://mlsanity.com/paper/2412.20066/) (2024)
- [Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity](https://mlsanity.com/paper/2203.11509/) (2022)
- [Rethinking Image Deraining via Rain Streaks and Vapors](https://mlsanity.com/paper/2008.00823/) (2020)
- [Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal](https://mlsanity.com/paper/2103.07051/) (2021)
- [Unsupervised Single Image Deraining with Self-supervised Constraints](https://mlsanity.com/paper/1811.08575/) (2018)
- [Dual-Path Coupled Image Deraining Network via Spatial-Frequency Interaction](https://mlsanity.com/paper/2402.04855/) (2024)
- [RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining](https://mlsanity.com/paper/2107.06808/) (2021)

## Built on by (most cited first, 1 of 1)

- [DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining](https://mlsanity.com/paper/2509.21719/) (2025)

## Abstract (arXiv)

Current image de-raining methods primarily learn from a limited dataset, leading to inadequate performance in varied real-world rainy conditions. To tackle this, we introduce a new framework that enables networks to progressively expand their de-raining knowledge base by tapping into a growing pool of datasets, significantly boosting their adaptability. Drawing inspiration from the human brain's ability to continuously absorb and generalize from ongoing experiences, our approach borrow the mechanism of the complementary learning system. Specifically, we first deploy Generative Adversarial Networks (GANs) to capture and retain the unique features of new data, mirroring the hippocampus's role in learning and memory. Then, the de-raining network is trained with both existing and GAN-synthesized data, mimicking the process of hippocampal replay and interleaved learning. Furthermore, we employ knowledge distillation with the replayed data to replicate the synergy between the neocortex's activity patterns triggered by hippocampal replays and the pre-existing neocortical knowledge. This comprehensive framework empowers the de-raining network to amass knowledge from various datasets, continually enhancing its performance on previously unseen rainy scenes. Our testing on three benchmark de-raining networks confirms the framework's effectiveness. It not only facilitates continuous knowledge accumulation across six datasets but also surpasses state-of-the-art methods in generalizing to new real-world scenarios.