# Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation

arXiv: https://arxiv.org/abs/2005.14553 · HTML: https://mlsanity.com/paper/2005.14553/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2020.

## Built on by (most cited first, 30 of 86)

- [MIC: Masked Image Consistency for Context-Enhanced Domain Adaptation](https://mlsanity.com/paper/2212.01322/) (2022)
- [SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.08808/) (2022)
- [DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation](https://mlsanity.com/paper/2104.10834/) (2021)
- [Unsupervised Domain Adaptation for Nighttime Aerial Tracking](https://mlsanity.com/paper/2203.10541/) (2022)
- [Flare7K: A Phenomenological Nighttime Flare Removal Dataset](https://mlsanity.com/paper/2210.06570/) (2022)
- [Zero-Shot Day-Night Domain Adaptation with a Physics Prior](https://mlsanity.com/paper/2108.05137/) (2021)
- [PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation](https://mlsanity.com/paper/2211.07609/) (2022)
- [Cross-Domain Correlation Distillation for Unsupervised Domain Adaptation in Nighttime Semantic Segmentation](https://mlsanity.com/paper/2205.00858/) (2022)
- [STEPS: Joint Self-supervised Nighttime Image Enhancement and Depth Estimation](https://mlsanity.com/paper/2302.01334/) (2023)
- [MUSES: The Multi-Sensor Semantic Perception Dataset for Driving under Uncertainty](https://mlsanity.com/paper/2401.12761/) (2024)
- [Refign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions](https://mlsanity.com/paper/2207.06825/) (2022)
- [Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond](https://mlsanity.com/paper/2306.04236/) (2023)
- [Prompting Diffusion Representations for Cross-Domain Semantic Segmentation](https://mlsanity.com/paper/2307.02138/) (2023)
- [CMDA: Cross-Modality Domain Adaptation for Nighttime Semantic Segmentation](https://mlsanity.com/paper/2307.15942/) (2023)
- [Self-Adversarial Disentangling for Specific Domain Adaptation](https://mlsanity.com/paper/2108.03553/) (2021)
- [NeRF-Loc: Visual Localization with Conditional Neural Radiance Field](https://mlsanity.com/paper/2304.07979/) (2023)
- [Domain Adaptive and Generalizable Network Architectures and Training Strategies for Semantic Image Segmentation](https://mlsanity.com/paper/2304.13615/) (2023)
- [MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks](https://mlsanity.com/paper/2203.01437/) (2022)
- [Tracker Meets Night: A Transformer Enhancer for UAV Tracking](https://mlsanity.com/paper/2303.10951/) (2023)
- [Learning with Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision](https://mlsanity.com/paper/2112.04719/) (2021)
- [ACDC: The Adverse Conditions Dataset with Correspondences for Robust Semantic Driving Scene Perception](https://mlsanity.com/paper/2104.13395/) (2021)
- [NightLab: A Dual-level Architecture with Hardness Detection for Segmentation at Night](https://mlsanity.com/paper/2204.05538/) (2022)
- [GDIP: Gated Differentiable Image Processing for Object-Detection in Adverse Conditions](https://mlsanity.com/paper/2209.14922/) (2022)
- [Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition](https://mlsanity.com/paper/2303.04291/) (2023)
- [Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillation](https://mlsanity.com/paper/2203.11160/) (2022)
- [Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters](https://mlsanity.com/paper/2207.01331/) (2022)
- [Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators](https://mlsanity.com/paper/2212.11237/) (2022)
- [A Survey on Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2302.09899/) (2023)
- [Unsupervised Domain Adaptation for Semantic Segmentation with Pseudo Label Self-Refinement](https://mlsanity.com/paper/2310.16979/) (2023)
- [Training-Free Model Merging for Multi-target Domain Adaptation](https://mlsanity.com/paper/2407.13771/) (2024)

## Abstract (arXiv)

We address the problem of semantic nighttime image segmentation and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night through progressively darker times of day, exploiting cross-time-of-day correspondences between daytime images from a reference map and dark images to guide the label inference in the dark domains; 2) a novel uncertainty-aware annotation and evaluation framework and metric for semantic segmentation, including image regions beyond human recognition capability in the evaluation in a principled fashion; 3) the Dark Zurich dataset, comprising 2416 unlabeled nighttime and 2920 unlabeled twilight images with correspondences to their daytime counterparts plus a set of 201 nighttime images with fine pixel-level annotations created with our protocol, which serves as a first benchmark for our novel evaluation. Experiments show that our map-guided curriculum adaptation significantly outperforms state-of-the-art methods on nighttime sets both for standard metrics and our uncertainty-aware metric. Furthermore, our uncertainty-aware evaluation reveals that selective invalidation of predictions can improve results on data with ambiguous content such as our benchmark and profit safety-oriented applications involving invalid inputs.