# Unsupervised Domain Adaptation for Video Semantic Segmentation

arXiv: https://arxiv.org/abs/2107.11052 · HTML: https://mlsanity.com/paper/2107.11052/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 45 references that are in the map.
- First posted on arXiv 23 Jul 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision](https://mlsanity.com/paper/2004.07703/) (2020)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Instance Adaptive Self-Training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.12197/) (2020)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [Video Panoptic Segmentation](https://mlsanity.com/paper/2006.11339/) (2020)
- [Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation](https://mlsanity.com/paper/2003.08040/) (2020)
- [STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos](https://mlsanity.com/paper/2003.08429/) (2020)
- [Deep Video Inpainting](https://mlsanity.com/paper/1905.01639/) (2019)
- [Temporal Attentive Alignment for Large-Scale Video Domain Adaptation](https://mlsanity.com/paper/1907.12743/) (2019)
- [Semantic Video Segmentation by Gated Recurrent Flow Propagation](https://mlsanity.com/paper/1612.08871/) (2016)
- [Semantic Video CNNs through Representation Warping](https://mlsanity.com/paper/1708.03088/) (2017)
- [Efficient Semantic Video Segmentation with Per-frame Inference](https://mlsanity.com/paper/2002.11433/) (2020)
- [Low-Latency Video Semantic Segmentation](https://mlsanity.com/paper/1804.00389/) (2018)
- [Two-phase Pseudo Label Densification for Self-training based Domain Adaptation](https://mlsanity.com/paper/2012.04828/) (2020)
- [Clockwork Convnets for Video Semantic Segmentation](https://mlsanity.com/paper/1608.03609/) (2016)
- [Dynamic Video Segmentation Network](https://mlsanity.com/paper/1804.00931/) (2018)

## Built on by (most cited first, 9 of 9)

- [Per-Clip Video Object Segmentation](https://mlsanity.com/paper/2208.01924/) (2022)
- [Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2211.10412/) (2022)
- [Spatio-Temporal Pixel-Level Contrastive Learning-based Source-Free Domain Adaptation for Video Semantic Segmentation](https://mlsanity.com/paper/2303.14361/) (2023)
- [Unified Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2311.13254/) (2023)
- [Prompt-Driven Temporal Domain Adaptation for Nighttime UAV Tracking](https://mlsanity.com/paper/2409.18533/) (2024)
- [We're Not Using Videos Effectively: An Updated Domain Adaptive Video Segmentation Baseline](https://mlsanity.com/paper/2402.00868/) (2024)
- [Domain Adaptive Video Semantic Segmentation via Cross-Domain Moving Object Mixing](https://mlsanity.com/paper/2211.02307/) (2022)
- [Weak-to-Strong Compositional Learning from Generative Models for Language-based Object Detection](https://mlsanity.com/paper/2407.15296/) (2024)
- [Time2General: Learning Spatiotemporal Invariant Representations for Domain-Generalization Video Semantic Segmentation](https://mlsanity.com/paper/2602.09648/) (2026)

## Abstract (arXiv)

Unsupervised Domain Adaptation for semantic segmentation has gained immense popularity since it can transfer knowledge from simulation to real (Sim2Real) by largely cutting out the laborious per pixel labeling efforts at real. In this work, we present a new video extension of this task, namely Unsupervised Domain Adaptation for Video Semantic Segmentation. As it became easy to obtain large-scale video labels through simulation, we believe attempting to maximize Sim2Real knowledge transferability is one of the promising directions for resolving the fundamental data-hungry issue in the video. To tackle this new problem, we present a novel two-phase adaptation scheme. In the first step, we exhaustively distill source domain knowledge using supervised loss functions. Simultaneously, video adversarial training (VAT) is employed to align the features from source to target utilizing video context. In the second step, we apply video self-training (VST), focusing only on the target data. To construct robust pseudo labels, we exploit the temporal information in the video, which has been rarely explored in the previous image-based self-training approaches. We set strong baseline scores on 'VIPER to CityscapeVPS' adaptation scenario. We show that our proposals significantly outperform previous image-based UDA methods both on image-level (mIoU) and video-level (VPQ) evaluation metrics.