# Rain rendering for evaluating and improving robustness to bad weather

arXiv: https://arxiv.org/abs/2009.03683 · HTML: https://mlsanity.com/paper/2009.03683/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2020.

## Built on by (most cited first, 30 of 57)

- [3D Common Corruptions and Data Augmentation](https://mlsanity.com/paper/2203.01441/) (2022)
- [Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather](https://mlsanity.com/paper/2108.05249/) (2021)
- [LiDAR Snowfall Simulation for Robust 3D Object Detection](https://mlsanity.com/paper/2203.15118/) (2022)
- [An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions](https://mlsanity.com/paper/2204.08817/) (2022)
- [Self-supervised Monocular Depth Estimation: Let's Talk About The Weather](https://mlsanity.com/paper/2307.08357/) (2023)
- [Physical 3D Adversarial Attacks against Monocular Depth Estimation in Autonomous Driving](https://mlsanity.com/paper/2403.17301/) (2024)
- [Refign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions](https://mlsanity.com/paper/2207.06825/) (2022)
- [ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering](https://mlsanity.com/paper/2305.02103/) (2023)
- [CoMoGAN: continuous model-guided image-to-image translation](https://mlsanity.com/paper/2103.06879/) (2021)
- [Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving](https://mlsanity.com/paper/2311.11056/) (2023)
- [MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks](https://mlsanity.com/paper/2203.01437/) (2022)
- [Online Domain Adaptation for Semantic Segmentation in Ever-Changing Conditions](https://mlsanity.com/paper/2207.10667/) (2022)
- [Perception Datasets for Anomaly Detection in Autonomous Driving: A Survey](https://mlsanity.com/paper/2302.02790/) (2023)
- [Distracting Downpour: Adversarial Weather Attacks for Motion Estimation](https://mlsanity.com/paper/2305.06716/) (2023)
- [WeatherDepth: Curriculum Contrastive Learning for Self-Supervised Depth Estimation under Adverse Weather Conditions](https://mlsanity.com/paper/2310.05556/) (2023)
- [To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2307.15063/) (2023)
- [HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation](https://mlsanity.com/paper/2403.12033/) (2024)
- [AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Driving](https://mlsanity.com/paper/2405.07865/) (2024)
- [InfraParis: A multi-modal and multi-task autonomous driving dataset](https://mlsanity.com/paper/2309.15751/) (2023)
- [WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion](https://mlsanity.com/paper/2504.13561/) (2025)
- [Controllable Weather Synthesis and Removal with Video Diffusion Models](https://mlsanity.com/paper/2505.00704/) (2025)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity](https://mlsanity.com/paper/2211.00837/) (2022)
- [ManiFest: Manifold Deformation for Few-shot Image Translation](https://mlsanity.com/paper/2111.13681/) (2021)
- [Understanding Time Variations of DNN Inference in Autonomous Driving](https://mlsanity.com/paper/2209.05487/) (2022)
- [Realistic Rainy Weather Simulation for LiDARs in CARLA Simulator](https://mlsanity.com/paper/2312.12772/) (2023)
- [EC-Depth: Exploring the consistency of self-supervised monocular depth estimation in challenging scenes](https://mlsanity.com/paper/2310.08044/) (2023)
- [Hybrid-TTA: Continual Test-time Adaptation via Dynamic Domain Shift Detection](https://mlsanity.com/paper/2409.08566/) (2024)
- [PRE-Mamba: A 4D State Space Model for Ultra-High-Frequent Event Camera Deraining](https://mlsanity.com/paper/2505.05307/) (2025)
- [StyleLess layer: Improving robustness for real-world driving](https://mlsanity.com/paper/2103.13905/) (2021)

## Abstract (arXiv)

Rain fills the atmosphere with water particles, which breaks the common assumption that light travels unaltered from the scene to the camera. While it is well-known that rain affects computer vision algorithms, quantifying its impact is difficult. In this context, we present a rain rendering pipeline that enables the systematic evaluation of common computer vision algorithms to controlled amounts of rain. We present three different ways to add synthetic rain to existing images datasets: completely physic-based; completely data-driven; and a combination of both. The physic-based rain augmentation combines a physical particle simulator and accurate rain photometric modeling. We validate our rendering methods with a user study, demonstrating our rain is judged as much as 73% more realistic than the state-of-theart. Using our generated rain-augmented KITTI, Cityscapes, and nuScenes datasets, we conduct a thorough evaluation of object detection, semantic segmentation, and depth estimation algorithms and show that their performance decreases in degraded weather, on the order of 15% for object detection, 60% for semantic segmentation, and 6-fold increase in depth estimation error. Finetuning on our augmented synthetic data results in improvements of 21% on object detection, 37% on semantic segmentation, and 8% on depth estimation.