# From Sky to the Ground: A Large-scale Benchmark and Simple Baseline Towards Real Rain Removal

arXiv: https://arxiv.org/abs/2308.03867 · HTML: https://mlsanity.com/paper/2308.03867/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [Towards End-to-End Lane Detection: an Instance Segmentation Approach](https://mlsanity.com/paper/1802.05591/) (2018)
- [A Model-driven Deep Neural Network for Single Image Rain Removal](https://mlsanity.com/paper/2005.01333/) (2020)
- [Single Image Deraining: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/1903.08558/) (2019)
- [Unpaired Deep Image Deraining Using Dual Contrastive Learning](https://mlsanity.com/paper/2109.02973/) (2021)
- [Toward Real-world Single Image Deraining: A New Benchmark and Beyond](https://mlsanity.com/paper/2206.05514/) (2022)
- [MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution](https://mlsanity.com/paper/2011.04566/) (2020)
- [Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation](https://mlsanity.com/paper/2103.13660/) (2021)
- [Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining](https://mlsanity.com/paper/1808.01491/) (2018)
- [Not Just Streaks: Towards Ground Truth for Single Image Deraining](https://mlsanity.com/paper/2206.10779/) (2022)
- [Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity](https://mlsanity.com/paper/2203.11509/) (2022)
- [FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors](https://mlsanity.com/paper/1803.07487/) (2018)
- [Subjective and Objective De-raining Quality Assessment Towards Authentic Rain Image](https://mlsanity.com/paper/1909.11983/) (2019)
- [Robust Online Matrix Factorization for Dynamic Background Subtraction](https://mlsanity.com/paper/1705.10000/) (2017)

## Built on by (most cited first, 30 of 39)

- [I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing](https://mlsanity.com/paper/2408.14180/) (2024)
- [Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining](https://mlsanity.com/paper/2404.01547/) (2024)
- [Towards Unified Deep Image Deraining: A Survey and A New Benchmark](https://mlsanity.com/paper/2310.03535/) (2023)
- [RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers](https://mlsanity.com/paper/2506.02528/) (2025)
- [Visual-Instructed Degradation Diffusion for All-in-One Image Restoration](https://mlsanity.com/paper/2506.16960/) (2025)
- [NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results](https://mlsanity.com/paper/2504.12711/) (2025)
- [Multi-Agent Image Restoration](https://mlsanity.com/paper/2503.09403/) (2025)
- [Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration](https://mlsanity.com/paper/2404.00288/) (2024)
- [UVEB: A Large-scale Benchmark and Baseline Towards Real-World Underwater Video Enhancement](https://mlsanity.com/paper/2404.14542/) (2024)
- [UAV-Rain1k: A Benchmark for Raindrop Removal from UAV Aerial Imagery](https://mlsanity.com/paper/2402.05773/) (2024)
- [RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration](https://mlsanity.com/paper/2505.18047/) (2025)
- [UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity](https://mlsanity.com/paper/2412.20157/) (2024)
- [GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restoration](https://mlsanity.com/paper/2411.17687/) (2024)
- [Controllable Weather Synthesis and Removal with Video Diffusion Models](https://mlsanity.com/paper/2505.00704/) (2025)
- [Reversing Flow for Image Restoration](https://mlsanity.com/paper/2506.16961/) (2025)
- [Image Editing with Diffusion Models: A Survey](https://mlsanity.com/paper/2504.13226/) (2025)
- [Image Deraining with Frequency-Enhanced State Space Model](https://mlsanity.com/paper/2405.16470/) (2024)
- [EGVD: Event-Guided Video Deraining](https://mlsanity.com/paper/2309.17239/) (2023)
- [A Hybrid Transformer-Mamba Network for Single Image Deraining](https://mlsanity.com/paper/2409.00410/) (2024)
- [DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining](https://mlsanity.com/paper/2509.21719/) (2025)
- [OPERA: An Agent for Image Restoration with End-to-End Joint Planning-Execution Optimization](https://mlsanity.com/paper/2605.22104/) (2026)
- [Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution](https://mlsanity.com/paper/2606.28971/) (2026)
- [Global Modeling Matters: A Fast, Lightweight and Effective Baseline for Efficient Image Restoration](https://mlsanity.com/paper/2507.13663/) (2025)
- [UniRain: Unified Image Deraining with RAG-based Dataset Distillation and Multi-objective Reweighted Optimization](https://mlsanity.com/paper/2603.03967/) (2026)
- [From Controlled Scenarios to Real-World: Cross-Domain Degradation Pattern Matching for All-in-One Image Restoration](https://mlsanity.com/paper/2505.22284/) (2025)
- [Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation](https://mlsanity.com/paper/2510.09228/) (2025)
- [Derain-Agent: A Plug-and-Play Agent Framework for Rainy Image Restoration](https://mlsanity.com/paper/2603.11866/) (2026)
- [UENR-600K: A Large-Scale Physically Grounded Dataset for Nighttime Video Deraining](https://mlsanity.com/paper/2604.04402/) (2026)
- [SkillIR: Evolving Scene-Aware Skills for Agentic Image Restoration](https://mlsanity.com/paper/2609.21468/) (2026)
- [TRG-Net: An Interpretable and Controllable Rain Generator](https://mlsanity.com/paper/2403.09993/) (2024)

## Abstract (arXiv)

Learning-based image deraining methods have made great progress. However, the lack of large-scale high-quality paired training samples is the main bottleneck to hamper the real image deraining (RID). To address this dilemma and advance RID, we construct a Large-scale High-quality Paired real rain benchmark (LHP-Rain), including 3000 video sequences with 1 million high-resolution (1920*1080) frame pairs. The advantages of the proposed dataset over the existing ones are three-fold: rain with higher-diversity and larger-scale, image with higher-resolution and higher-quality ground-truth. Specifically, the real rains in LHP-Rain not only contain the classical rain streak/veiling/occlusion in the sky, but also the splashing on the ground overlooked by deraining community. Moreover, we propose a novel robust low-rank tensor recovery model to generate the GT with better separating the static background from the dynamic rain. In addition, we design a simple transformer-based single image deraining baseline, which simultaneously utilize the self-attention and cross-layer attention within the image and rain layer with discriminative feature representation. Extensive experiments verify the superiority of the proposed dataset and deraining method over state-of-the-art.