# Networks are Slacking Off: Understanding Generalization Problem in Image Deraining

arXiv: https://arxiv.org/abs/2305.15134 · HTML: https://mlsanity.com/paper/2305.15134/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 77 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 30 of 54)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Blind Super-Resolution With Iterative Kernel Correction](https://mlsanity.com/paper/1904.03377/) (2019)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration](https://mlsanity.com/paper/2007.12142/) (2020)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [Clearing the Skies: A deep network architecture for single-image rain removal](https://mlsanity.com/paper/1609.02087/) (2016)
- [Interpreting Super-Resolution Networks with Local Attribution Maps](https://mlsanity.com/paper/2011.11036/) (2020)
- [A Model-driven Deep Neural Network for Single Image Rain Removal](https://mlsanity.com/paper/2005.01333/) (2020)

## Built on by (most cited first, 14 of 14)

- [Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild](https://mlsanity.com/paper/2401.13627/) (2024)
- [An Intelligent Agentic System for Complex Image Restoration Problems](https://mlsanity.com/paper/2410.17809/) (2024)
- [Crafting Training Degradation Distribution for the Accuracy-Generalization Trade-off in Real-World Super-Resolution](https://mlsanity.com/paper/2305.18107/) (2023)
- [Multi-Agent Image Restoration](https://mlsanity.com/paper/2503.09403/) (2025)
- [Channel Consistency Prior and Self-Reconstruction Strategy Based Unsupervised Image Deraining](https://mlsanity.com/paper/2503.18703/) (2025)
- [PhotoArtAgent: Intelligent Photo Retouching with Language Model-Based Artist Agents](https://mlsanity.com/paper/2505.23130/) (2025)
- [Interpreting Low-level Vision Models with Causal Effect Maps](https://mlsanity.com/paper/2407.19789/) (2024)
- [CasSR: Activating Image Power for Real-World Image Super-Resolution](https://mlsanity.com/paper/2403.11451/) (2024)
- [A Preliminary Exploration Towards General Image Restoration](https://mlsanity.com/paper/2408.15143/) (2024)
- [Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining](https://mlsanity.com/paper/2502.12600/) (2025)
- [BurstDeflicker: A Benchmark Dataset for Flicker Removal in Dynamic Scenes](https://mlsanity.com/paper/2510.09996/) (2025)
- [Position: Agentic Systems Constitute a Key Component of Next-Generation Intelligent Image Processing](https://mlsanity.com/paper/2505.16007/) (2025)
- [ConStyle v2: A Strong Prompter for All-in-One Image Restoration](https://mlsanity.com/paper/2406.18242/) (2024)
- [How far have we gone in Generative Image Restoration? A study on its capability, limitations and evaluation practices](https://mlsanity.com/paper/2603.05010/) (2026)

## Abstract (arXiv)

Deep deraining networks consistently encounter substantial generalization issues when deployed in real-world applications, although they are successful in laboratory benchmarks. A prevailing perspective in deep learning encourages using highly complex data for training, with the expectation that richer image background content will facilitate overcoming the generalization problem. However, through comprehensive and systematic experimentation, we discover that this strategy does not enhance the generalization capability of these networks. On the contrary, it exacerbates the tendency of networks to overfit specific degradations. Our experiments reveal that better generalization in a deraining network can be achieved by simplifying the complexity of the training background images. This is because that the networks are ``slacking off'' during training, that is, learning the least complex elements in the image background and degradation to minimize training loss. When the background images are less complex than the rain streaks, the network will prioritize the background reconstruction, thereby suppressing overfitting the rain patterns and leading to improved generalization performance. Our research offers a valuable perspective and methodology for better understanding the generalization problem in low-level vision tasks and displays promising potential for practical application.