# TRG-Net: An Interpretable and Controllable Rain Generator

arXiv: https://arxiv.org/abs/2403.09993 · HTML: https://mlsanity.com/paper/2403.09993/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 69 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Learning Steerable Filters for Rotation Equivariant CNNs](https://mlsanity.com/paper/1711.07289/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [A Model-driven Deep Neural Network for Single Image Rain Removal](https://mlsanity.com/paper/2005.01333/) (2020)
- [Single Image Deraining: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/1903.08558/) (2019)
- [Single Image Deraining: From Model-Based to Data-Driven and Beyond](https://mlsanity.com/paper/1912.07150/) (2019)
- [Lightweight Pyramid Networks for Image Deraining](https://mlsanity.com/paper/1805.06173/) (2018)
- [Syn2Real Transfer Learning for Image Deraining using Gaussian Processes](https://mlsanity.com/paper/2006.05580/) (2020)
- [Unpaired Deep Image Deraining Using Dual Contrastive Learning](https://mlsanity.com/paper/2109.02973/) (2021)
- [Rain rendering for evaluating and improving robustness to bad weather](https://mlsanity.com/paper/2009.03683/) (2020)
- [Physics-Based Rendering for Improving Robustness to Rain](https://mlsanity.com/paper/1908.10335/) (2019)
- [Toward Real-world Single Image Deraining: A New Benchmark and Beyond](https://mlsanity.com/paper/2206.05514/) (2022)
- [From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal](https://mlsanity.com/paper/2308.03867/) (2023)
- [Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity](https://mlsanity.com/paper/2203.11509/) (2022)
- [Fourier Series Expansion Based Filter Parametrization for Equivariant Convolutions](https://mlsanity.com/paper/2107.14519/) (2021)
- [A Survey on Rain Removal from Video and Single Image](https://mlsanity.com/paper/1909.08326/) (2019)
- [RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining](https://mlsanity.com/paper/2107.06808/) (2021)

## Abstract (arXiv)

Exploring and modeling rain generation mechanism is critical for augmenting paired data to ease training of rainy image processing models. Against this task, this study proposes a novel deep learning based rain generator, which fully takes the physical generation mechanism underlying rains into consideration and well encodes the learning of the fundamental rain factors (i.e., shape, orientation, length, width and sparsity) explicitly into the deep network. Its significance lies in that the generator not only elaborately design essential elements of the rain to simulate expected rains, like conventional artificial strategies, but also finely adapt to complicated and diverse practical rainy images, like deep learning methods. By rationally adopting filter parameterization technique, we first time achieve a deep network that is finely controllable with respect to rain factors and able to learn the distribution of these factors purely from data. Our unpaired generation experiments demonstrate that the rain generated by the proposed rain generator is not only of higher quality, but also more effective for deraining and downstream tasks compared to current state-of-the-art rain generation methods. Besides, the paired data augmentation experiments, including both in-distribution and out-of-distribution (OOD), further validate the diversity of samples generated by our model for in-distribution deraining and OOD generalization tasks.