# Single Image Deraining Network with Rain Embedding Consistency and Layered LSTM

arXiv: https://arxiv.org/abs/2111.03615 · HTML: https://mlsanity.com/paper/2111.03615/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 5 Nov 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [A Model-driven Deep Neural Network for Single Image Rain Removal](https://mlsanity.com/paper/2005.01333/) (2020)
- [Single Image Deraining: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/1903.08558/) (2019)
- [Single Image Deraining: From Model-Based to Data-Driven and Beyond](https://mlsanity.com/paper/1912.07150/) (2019)
- [DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal](https://mlsanity.com/paper/2008.00767/) (2020)
- [Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining](https://mlsanity.com/paper/1808.01491/) (2018)
- [Robust Representation Learning with Feedback for Single Image Deraining](https://mlsanity.com/paper/2101.12463/) (2021)
- [Recurrent U-Net for Resource-Constrained Segmentation](https://mlsanity.com/paper/1906.04913/) (2019)
- [Joint Self-Attention and Scale-Aggregation for Self-Calibrated Deraining Network](https://mlsanity.com/paper/2008.02763/) (2020)
- [Conditional Variational Image Deraining](https://mlsanity.com/paper/2004.11373/) (2020)
- [Residual-Guide Feature Fusion Network for Single Image Deraining](https://mlsanity.com/paper/1804.07493/) (2018)
- [DRD-Net: Detail-recovery Image Deraining via Context Aggregation Networks](https://mlsanity.com/paper/1908.10267/) (2019)
- [Confidence Measure Guided Single Image De-raining](https://mlsanity.com/paper/1909.04207/) (2019)

## Built on by (most cited first, 4 of 4)

- [Dual-Pixel Raindrop Removal](https://mlsanity.com/paper/2210.13321/) (2022)
- [An End-to-End Cascaded Image Deraining and Object Detection Neural Network](https://mlsanity.com/paper/2202.11279/) (2022)
- [Rethinking Rainy 3D Scene Reconstruction via Perspective Transforming and Brightness Tuning](https://mlsanity.com/paper/2511.06734/) (2025)
- [Uncertainty-Driven Multi-Scale Feature Fusion Network for Real-time Image Deraining](https://mlsanity.com/paper/2307.09728/) (2023)

## Abstract (arXiv)

Single image deraining is typically addressed as residual learning to predict the rain layer from an input rainy image. For this purpose, an encoder-decoder network draws wide attention, where the encoder is required to encode a high-quality rain embedding which determines the performance of the subsequent decoding stage to reconstruct the rain layer. However, most of existing studies ignore the significance of rain embedding quality, thus leading to limited performance with over/under-deraining. In this paper, with our observation of the high rain layer reconstruction performance by an rain-to-rain autoencoder, we introduce the idea of "Rain Embedding Consistency" by regarding the encoded embedding by the autoencoder as an ideal rain embedding and aim at enhancing the deraining performance by improving the consistency between the ideal rain embedding and the rain embedding derived by the encoder of the deraining network. To achieve this, a Rain Embedding Loss is applied to directly supervise the encoding process, with a Rectified Local Contrast Normalization (RLCN) as the guide that effectively extracts the candidate rain pixels. We also propose Layered LSTM for recurrent deraining and fine-grained encoder feature refinement considering different scales. Qualitative and quantitative experiments demonstrate that our proposed method outperforms previous state-of-the-art methods particularly on a real-world dataset. Our source code is available at http://www.ok.sc.e.titech.ac.jp/res/SIR/.