# Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation

arXiv: https://arxiv.org/abs/2105.02001 · HTML: https://mlsanity.com/paper/2105.02001/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 52 references that are in the map.
- First posted on arXiv 5 May 2021.

## Builds on (most cited first, 30 of 40)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Unified Deep Supervised Domain Adaptation and Generalization](https://mlsanity.com/paper/1709.10190/) (2017)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation](https://mlsanity.com/paper/1809.09478/) (2018)
- [FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation](https://mlsanity.com/paper/1612.02649/) (2016)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [DACS: Domain Adaptation via Cross-domain Mixed Sampling](https://mlsanity.com/paper/2007.08702/) (2020)
- [Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2003.03773/) (2020)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [Instance Adaptive Self-Training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.12197/) (2020)
- [Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1705.00609/) (2017)
- [Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/1910.13049/) (2019)

## Built on by (most cited first, 19 of 19)

- [Robust Mean Teacher for Continual and Gradual Test-Time Adaptation](https://mlsanity.com/paper/2211.13081/) (2022)
- [Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correction](https://mlsanity.com/paper/2306.00650/) (2023)
- [Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2207.06654/) (2022)
- [Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving](https://mlsanity.com/paper/2304.11928/) (2023)
- [Generalization by Adaptation: Diffusion-Based Domain Extension for Domain-Generalized Semantic Segmentation](https://mlsanity.com/paper/2312.01850/) (2023)
- [DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment](https://mlsanity.com/paper/2405.11765/) (2024)
- [Introducing Intermediate Domains for Effective Self-Training during Test-Time](https://mlsanity.com/paper/2208.07736/) (2022)
- [Contrastive Model Adaptation for Cross-Condition Robustness in Semantic Segmentation](https://mlsanity.com/paper/2303.05194/) (2023)
- [Contrast, Stylize and Adapt: Unsupervised Contrastive Learning Framework for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2306.09098/) (2023)
- [Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer](https://mlsanity.com/paper/2208.06507/) (2022)
- [Unsupervised Contrastive Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2204.08399/) (2022)
- [Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation](https://mlsanity.com/paper/2302.06992/) (2023)
- [An Active and Contrastive Learning Framework for Fine-Grained Off-Road Semantic Segmentation](https://mlsanity.com/paper/2202.09002/) (2022)
- [Resolution-Aware Design of Atrous Rates for Semantic Segmentation Networks](https://mlsanity.com/paper/2307.14179/) (2023)
- [A Study on Unsupervised Domain Adaptation for Semantic Segmentation in the Era of Vision-Language Models](https://mlsanity.com/paper/2411.16407/) (2024)
- [DCFS: Continual Test-Time Adaptation via Dual Consistency of Feature and Sample](https://mlsanity.com/paper/2508.20516/) (2025)
- [SynthGenNet: a self-supervised approach for test-time generalization using synthetic multi-source domain mixing of street view images](https://mlsanity.com/paper/2509.02287/) (2025)
- [Unsupervised Domain Adaptation via Similarity-based Prototypes for Cross-Modality Segmentation](https://mlsanity.com/paper/2510.20596/) (2025)
- [Structured prototype regularization for synthetic-to-real driving scene parsing](https://mlsanity.com/paper/2603.16083/) (2026)

## Abstract (arXiv)

Deep convolutional neural networks have considerably improved state-of-the-art results for semantic segmentation. Nevertheless, even modern architectures lack the ability to generalize well to a test dataset that originates from a different domain. To avoid the costly annotation of training data for unseen domains, unsupervised domain adaptation (UDA) attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. Previous work has mainly focused on minimizing the discrepancy between the two domains by using adversarial training or self-training. While adversarial training may fail to align the correct semantic categories as it minimizes the discrepancy between the global distributions, self-training raises the question of how to provide reliable pseudo-labels. To align the correct semantic categories across domains, we propose a contrastive learning approach that adapts category-wise centroids across domains. Furthermore, we extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels. Although both contrastive learning and self-training (CLST) through temporal ensembling enable knowledge transfer between two domains, it is their combination that leads to a symbiotic structure. We validate our approach on two domain adaptation benchmarks: GTA5 rightarrow Cityscapes and SYNTHIA rightarrow Cityscapes. Our method achieves better or comparable results than the state-of-the-art. We will make the code publicly available.