# Domain Adaptive Video Semantic Segmentation via Cross-Domain Moving Object Mixing

arXiv: https://arxiv.org/abs/2211.02307 · HTML: https://mlsanity.com/paper/2211.02307/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 40 references that are in the map.
- First posted on arXiv 4 Nov 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2101.10979/) (2021)
- [FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation](https://mlsanity.com/paper/1612.02649/) (2016)
- [DACS: Domain Adaptation via Cross-domain Mixed Sampling](https://mlsanity.com/paper/2007.08702/) (2020)
- [ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learning](https://mlsanity.com/paper/2007.07936/) (2020)
- [Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision](https://mlsanity.com/paper/2004.07703/) (2020)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Instance Adaptive Self-Training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.12197/) (2020)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1909.00589/) (2019)
- [Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation](https://mlsanity.com/paper/2003.08040/) (2020)
- [Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video](https://mlsanity.com/paper/1807.06667/) (2018)
- [Semantic Video Segmentation by Gated Recurrent Flow Propagation](https://mlsanity.com/paper/1612.08871/) (2016)
- [Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2007.02424/) (2020)
- [Semantic Video CNNs through Representation Warping](https://mlsanity.com/paper/1708.03088/) (2017)
- [Clockwork Convnets for Video Semantic Segmentation](https://mlsanity.com/paper/1608.03609/) (2016)
- [RDA: Robust Domain Adaptation via Fourier Adversarial Attacking](https://mlsanity.com/paper/2106.02874/) (2021)
- [Unsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings](https://mlsanity.com/paper/2011.12616/) (2020)

## Built on by (most cited first, 3 of 3)

- [Unified Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2311.13254/) (2023)
- [We're Not Using Videos Effectively: An Updated Domain Adaptive Video Segmentation Baseline](https://mlsanity.com/paper/2402.00868/) (2024)
- [PiPa++: Towards Unification of Domain Adaptive Semantic Segmentation via Self-supervised Learning](https://mlsanity.com/paper/2407.17101/) (2024)

## Abstract (arXiv)

The network trained for domain adaptation is prone to bias toward the easy-to-transfer classes. Since the ground truth label on the target domain is unavailable during training, the bias problem leads to skewed predictions, forgetting to predict hard-to-transfer classes. To address this problem, we propose Cross-domain Moving Object Mixing (CMOM) that cuts several objects, including hard-to-transfer classes, in the source domain video clip and pastes them into the target domain video clip. Unlike image-level domain adaptation, the temporal context should be maintained to mix moving objects in two different videos. Therefore, we design CMOM to mix with consecutive video frames, so that unrealistic movements are not occurring. We additionally propose Feature Alignment with Temporal Context (FATC) to enhance target domain feature discriminability. FATC exploits the robust source domain features, which are trained with ground truth labels, to learn discriminative target domain features in an unsupervised manner by filtering unreliable predictions with temporal consensus. We demonstrate the effectiveness of the proposed approaches through extensive experiments. In particular, our model reaches mIoU of 53.81% on VIPER to Cityscapes-Seq benchmark and mIoU of 56.31% on SYNTHIA-Seq to Cityscapes-Seq benchmark, surpassing the state-of-the-art methods by large margins. The code is available at: https://github.com/kyusik-cho/CMOM.