# Semantic Segmentation under Adverse Conditions: A Weather and Nighttime-aware Synthetic Data-based Approach

arXiv: https://arxiv.org/abs/2210.05626 · HTML: https://mlsanity.com/paper/2210.05626/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation](https://mlsanity.com/paper/2206.08367/) (2022)
- [Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing](https://mlsanity.com/paper/1810.08705/) (2018)
- [MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving](https://mlsanity.com/paper/1612.07695/) (2016)
- [Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.12940/) (2021)
- [SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2008.04017/) (2020)
- [Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding](https://mlsanity.com/paper/2112.00484/) (2021)
- [Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications](https://mlsanity.com/paper/1710.06270/) (2017)
- [Competitive Simplicity for Multi-Task Learning for Real-Time Foggy Scene Understanding via Domain Adaptation](https://mlsanity.com/paper/2012.05304/) (2020)
- [DSNet: An Efficient CNN for Road Scene Segmentation](https://mlsanity.com/paper/1904.05022/) (2019)
- [Dataset Augmentation with Synthetic Images Improves Semantic Segmentation](https://mlsanity.com/paper/1709.00849/) (2017)

## Built on by (most cited first, 7 of 7)

- [Aleth-NeRF: Low-light Condition View Synthesis with Concealing Fields](https://mlsanity.com/paper/2303.05807/) (2023)
- [WeatherProof: A Paired-Dataset Approach to Semantic Segmentation in Adverse Weather](https://mlsanity.com/paper/2312.09534/) (2023)
- [CaRaFFusion: Improving 2D Semantic Segmentation with Camera-Radar Point Cloud Fusion and Zero-Shot Image Inpainting](https://mlsanity.com/paper/2505.03679/) (2025)
- [WeatherProof: Leveraging Language Guidance for Semantic Segmentation in Adverse Weather](https://mlsanity.com/paper/2403.14874/) (2024)
- [RobuMTL: Enhancing Multi-Task Learning Robustness Against Weather Conditions](https://mlsanity.com/paper/2601.10921/) (2026)
- [Solution for CVPR 2024 UG2+ Challenge Track on All Weather Semantic Segmentation](https://mlsanity.com/paper/2406.05837/) (2024)
- [AVOID: The Adverse Visual Conditions Dataset with Obstacles for Driving Scene Understanding](https://mlsanity.com/paper/2512.23215/) (2025)

## Abstract (arXiv)

Recent semantic segmentation models perform well under standard weather conditions and sufficient illumination but struggle with adverse weather conditions and nighttime. Collecting and annotating training data under these conditions is expensive, time-consuming, error-prone, and not always practical. Usually, synthetic data is used as a feasible data source to increase the amount of training data. However, just directly using synthetic data may actually harm the model's performance under normal weather conditions while getting only small gains in adverse situations. Therefore, we present a novel architecture specifically designed for using synthetic training data for domain adaptation. We propose a simple yet powerful addition to DeepLabV3+ by using weather and time-of-the-day supervisors trained with multi-task learning, making it both weather and nighttime aware, which improves its mIoU accuracy by 14 percentage points on the ACDC dataset while maintaining a score of 75% mIoU on the Cityscapes dataset. Our code is available at https://github.com/lsmcolab/Semantic-Segmentation-under-Adverse-Conditions.