# Reducing Annotation Effort by Identifying and Labeling Contextually Diverse Classes for Semantic Segmentation Under Domain Shift

arXiv: https://arxiv.org/abs/2210.06749 · HTML: https://mlsanity.com/paper/2210.06749/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 47 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 30 of 35)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019)
- [Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation](https://mlsanity.com/paper/1809.09478/) (2018)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2101.10979/) (2021)
- [Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision](https://mlsanity.com/paper/2004.07703/) (2020)
- [Instance Adaptive Self-Training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.12197/) (2020)
- [Source-Free Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16372/) (2021)
- [Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/1910.13049/) (2019)
- [Domain Adaptation for Structured Output via Discriminative Patch Representations](https://mlsanity.com/paper/1901.05427/) (2019)
- [Contextual Diversity for Active Learning](https://mlsanity.com/paper/2008.05723/) (2020)
- [Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach](https://mlsanity.com/paper/1908.09547/) (2019)
- [Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.11249/) (2021)
- [Active Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.07848/) (2019)
- [Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings](https://mlsanity.com/paper/2010.08666/) (2020)
- [Domain Adaptive Semantic Segmentation Using Weak Labels](https://mlsanity.com/paper/2007.15176/) (2020)
- [Source-Relaxed Domain Adaptation for Image Segmentation](https://mlsanity.com/paper/2005.03697/) (2020)
- [Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation](https://mlsanity.com/paper/2004.00794/) (2020)
- [Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation](https://mlsanity.com/paper/2103.04705/) (2021)

## Abstract (arXiv)

In Active Domain Adaptation (ADA), one uses Active Learning (AL) to select a subset of images from the target domain, which are then annotated and used for supervised domain adaptation (DA). Given the large performance gap between supervised and unsupervised DA techniques, ADA allows for an excellent trade-off between annotation cost and performance. Prior art makes use of measures of uncertainty or disagreement of models to identify `regions' to be annotated by the human oracle. However, these regions frequently comprise of pixels at object boundaries which are hard and tedious to annotate. Hence, even if the fraction of image pixels annotated reduces, the overall annotation time and the resulting cost still remain high. In this work, we propose an ADA strategy, which given a frame, identifies a set of classes that are hardest for the model to predict accurately, thereby recommending semantically meaningful regions to be annotated in a selected frame. We show that these set of `hard' classes are context-dependent and typically vary across frames, and when annotated help the model generalize better. We propose two ADA techniques: the Anchor-based and Augmentation-based approaches to select complementary and diverse regions in the context of the current training set. Our approach achieves 66.6 mIoU on GTA to Cityscapes dataset with an annotation budget of 4.7% in comparison to 64.9 mIoU by MADA using 5% of annotations. Our technique can also be used as a decorator for any existing frame-based AL technique, e.g., we report 1.5% performance improvement for CDAL on Cityscapes using our approach.