# Coarse-to-Fine Domain Adaptive Semantic Segmentation with Photometric Alignment and Category-Center Regularization

arXiv: https://arxiv.org/abs/2103.13041 · HTML: https://mlsanity.com/paper/2103.13041/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation](https://mlsanity.com/paper/1809.09478/) (2018)
- [FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation](https://mlsanity.com/paper/1612.02649/) (2016)
- [Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation](https://mlsanity.com/paper/1803.11365/) (2018)
- [Conditional Random Fields as Recurrent Neural Networks](https://mlsanity.com/paper/1502.03240/) (2015)
- [Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision](https://mlsanity.com/paper/2004.07703/) (2020)
- [Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/1910.13049/) (2019)
- [Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation](https://mlsanity.com/paper/2007.09222/) (2020)
- [Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation](https://mlsanity.com/paper/2003.08040/) (2020)
- [No More Discrimination: Cross City Adaptation of Road Scene Segmenters](https://mlsanity.com/paper/1704.08509/) (2017)
- [DCAN: Dual Channel-wise Alignment Networks for Unsupervised Scene Adaptation](https://mlsanity.com/paper/1804.05827/) (2018)
- [Self-Training and Adversarial Background Regularization for Unsupervised Domain Adaptive One-Stage Object Detection](https://mlsanity.com/paper/1909.00597/) (2019)
- [TSIT: A Simple and Versatile Framework for Image-to-Image Translation](https://mlsanity.com/paper/2007.12072/) (2020)
- [3D Whole Brain Segmentation using Spatially Localized Atlas Network Tiles](https://mlsanity.com/paper/1903.12152/) (2019)
- [Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2007.07695/) (2020)
- [Unsupervised Deep Learning for Bayesian Brain MRI Segmentation](https://mlsanity.com/paper/1904.11319/) (2019)

## Built on by (most cited first, 30 of 36)

- [DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2111.14887/) (2021)
- [HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.13132/) (2022)
- [SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.08808/) (2022)
- [WildNet: Learning Domain Generalized Semantic Segmentation from the Wild](https://mlsanity.com/paper/2204.01446/) (2022)
- [Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation](https://mlsanity.com/paper/2203.01452/) (2022)
- [Source-Free Open Compound Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2106.03422/) (2021)
- [ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation](https://mlsanity.com/paper/2111.15242/) (2021)
- [Cross-Domain Correlation Distillation for Unsupervised Domain Adaptation in Nighttime Semantic Segmentation](https://mlsanity.com/paper/2205.00858/) (2022)
- [Deliberated Domain Bridging for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2209.07695/) (2022)
- [Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving](https://mlsanity.com/paper/2304.11928/) (2023)
- [Diffusion-based Image Translation with Label Guidance for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2308.12350/) (2023)
- [SATS: Self-Attention Transfer for Continual Semantic Segmentation](https://mlsanity.com/paper/2203.07667/) (2022)
- [Learning from Future: A Novel Self-Training Framework for Semantic Segmentation](https://mlsanity.com/paper/2209.06993/) (2022)
- [Smoothing Matters: Momentum Transformer for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2203.07988/) (2022)
- [Contrast, Stylize and Adapt: Unsupervised Contrastive Learning Framework for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2306.09098/) (2023)
- [ADAS: A Direct Adaptation Strategy for Multi-Target Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2203.06811/) (2022)
- [Bidirectional Self-Training with Multiple Anisotropic Prototypes for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.07730/) (2022)
- [Reducing Domain Gap in Frequency and Spatial domain for Cross-modality Domain Adaptation on Medical Image Segmentation](https://mlsanity.com/paper/2211.15235/) (2022)
- [ML-BPM: Multi-teacher Learning with Bidirectional Photometric Mixing for Open Compound Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2207.09045/) (2022)
- [I2F: A Unified Image-to-Feature Approach for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2301.01149/) (2023)
- [Informative Data Mining for One-Shot Cross-Domain Semantic Segmentation](https://mlsanity.com/paper/2309.14241/) (2023)
- [Unsupervised Contrastive Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2204.08399/) (2022)
- [Threshold-adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/2208.10716/) (2022)
- [IBAFormer: Intra-batch Attention Transformer for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2309.06282/) (2023)
- [Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation](https://mlsanity.com/paper/2302.06992/) (2023)
- [Geometry-Aware Network for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2212.00920/) (2022)
- [Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation](https://mlsanity.com/paper/2312.06331/) (2023)
- [Domain-invariant Prototypes for Semantic Segmentation](https://mlsanity.com/paper/2208.06087/) (2022)
- [Towards Dynamic and Small Objects Refinement for Unsupervised Domain Adaptative Nighttime Semantic Segmentation](https://mlsanity.com/paper/2310.04747/) (2023)
- [Beyond Deterministic Translation for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2202.07778/) (2022)

## Abstract (arXiv)

Unsupervised domain adaptation (UDA) in semantic segmentation is a fundamental yet promising task relieving the need for laborious annotation works. However, the domain shifts/discrepancies problem in this task compromise the final segmentation performance. Based on our observation, the main causes of the domain shifts are differences in imaging conditions, called image-level domain shifts, and differences in object category configurations called category-level domain shifts. In this paper, we propose a novel UDA pipeline that unifies image-level alignment and category-level feature distribution regularization in a coarse-to-fine manner. Specifically, on the coarse side, we propose a photometric alignment module that aligns an image in the source domain with a reference image from the target domain using a set of image-level operators; on the fine side, we propose a category-oriented triplet loss that imposes a soft constraint to regularize category centers in the source domain and a self-supervised consistency regularization method in the target domain. Experimental results show that our proposed pipeline improves the generalization capability of the final segmentation model and significantly outperforms all previous state-of-the-arts.