# Physics-Based Rendering for Improving Robustness to Rain

arXiv: https://arxiv.org/abs/1908.10335 · HTML: https://mlsanity.com/paper/1908.10335/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2019.

## Built on by (most cited first, 30 of 49)

- [NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation](https://mlsanity.com/paper/2208.05117/) (2022)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions](https://mlsanity.com/paper/2204.08817/) (2022)
- [Towards Robust Rain Removal Against Adversarial Attacks: A Comprehensive Benchmark Analysis and Beyond](https://mlsanity.com/paper/2203.16931/) (2022)
- [ActMAD: Activation Matching to Align Distributions for Test-Time-Training](https://mlsanity.com/paper/2211.12870/) (2022)
- [Not Just Streaks: Towards Ground Truth for Single Image Deraining](https://mlsanity.com/paper/2206.10779/) (2022)
- [Robustness of Object Detectors in Degrading Weather Conditions](https://mlsanity.com/paper/2106.08795/) (2021)
- [ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering](https://mlsanity.com/paper/2305.02103/) (2023)
- [A Simple Recipe for Language-guided Domain Generalized Segmentation](https://mlsanity.com/paper/2311.17922/) (2023)
- [CoMoGAN: continuous model-guided image-to-image translation](https://mlsanity.com/paper/2103.06879/) (2021)
- [When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes](https://mlsanity.com/paper/2104.02215/) (2021)
- [ACDC: The Adverse Conditions Dataset with Correspondences for Robust Semantic Driving Scene Perception](https://mlsanity.com/paper/2104.13395/) (2021)
- [Distracting Downpour: Adversarial Weather Attacks for Motion Estimation](https://mlsanity.com/paper/2305.06716/) (2023)
- [What, How, and When Should Object Detectors Update in Continually Changing Test Domains?](https://mlsanity.com/paper/2312.08875/) (2023)
- [HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation](https://mlsanity.com/paper/2403.12033/) (2024)
- [RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining](https://mlsanity.com/paper/2107.06808/) (2021)
- [RadSegNet: A Reliable Approach to Radar Camera Fusion](https://mlsanity.com/paper/2208.03849/) (2022)
- [Worsening Perception: Real-time Degradation of Autonomous Vehicle Perception Performance for Simulation of Adverse Weather Conditions](https://mlsanity.com/paper/2103.02760/) (2021)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [RaidaR: A Rich Annotated Image Dataset of Rainy Street Scenes](https://mlsanity.com/paper/2104.04606/) (2021)
- [Robust 3D Object Detection in Cold Weather Conditions](https://mlsanity.com/paper/2205.11925/) (2022)
- [Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity](https://mlsanity.com/paper/2211.00837/) (2022)
- [RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo](https://mlsanity.com/paper/2505.09368/) (2025)
- [Task-Driven Deep Image Enhancement Network for Autonomous Driving in Bad Weather](https://mlsanity.com/paper/2110.07206/) (2021)
- [SUSTechGAN: Image Generation for Object Detection in Adverse Conditions of Autonomous Driving](https://mlsanity.com/paper/2408.01430/) (2024)
- [ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media](https://mlsanity.com/paper/2102.05847/) (2021)
- [Simulating Road Spray Effects in Automotive Lidar Sensor Models](https://mlsanity.com/paper/2212.08558/) (2022)
- [Assessing Domain Gap for Continual Domain Adaptation in Object Detection](https://mlsanity.com/paper/2302.10396/) (2023)
- [Benchmarking Robustness of AI-Enabled Multi-sensor Fusion Systems: Challenges and Opportunities](https://mlsanity.com/paper/2306.03454/) (2023)
- [Weakly Supervised Test-Time Domain Adaptation for Object Detection](https://mlsanity.com/paper/2407.05607/) (2024)

## Abstract (arXiv)

To improve the robustness to rain, we present a physically-based rain rendering pipeline for realistically inserting rain into clear weather images. Our rendering relies on a physical particle simulator, an estimation of the scene lighting and an accurate rain photometric modeling to augment images with arbitrary amount of realistic rain or fog. We validate our rendering with a user study, proving our rain is judged 40% more realistic that state-of-the-art. Using our generated weather augmented Kitti and Cityscapes dataset, we conduct a thorough evaluation of deep object detection and semantic segmentation algorithms and show that their performance decreases in degraded weather, on the order of 15% for object detection and 60% for semantic segmentation. Furthermore, we show refining existing networks with our augmented images improves the robustness of both object detection and semantic segmentation algorithms. We experiment on nuScenes and measure an improvement of 15% for object detection and 35% for semantic segmentation compared to original rainy performance. Augmented databases and code are available on the project page.