# Urban Scene Semantic Segmentation with Low-Cost Coarse Annotation

arXiv: https://arxiv.org/abs/2212.07911 · HTML: https://mlsanity.com/paper/2212.07911/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 47 references that are in the map.
- First posted on arXiv 15 Dec 2022.

## Builds on (most cited first, 30 of 35)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data](https://mlsanity.com/paper/1909.00889/) (2019)
- [Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/1906.00817/) (2019)
- [Simple Does It: Weakly Supervised Instance and Semantic Segmentation](https://mlsanity.com/paper/1603.07485/) (2016)
- [DACS: Domain Adaptation via Cross-domain Mixed Sampling](https://mlsanity.com/paper/2007.08702/) (2020)
- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Adversarial Learning for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/1802.07934/) (2018)
- [Gated-SCNN: Gated Shape CNNs for Semantic Segmentation](https://mlsanity.com/paper/1907.05740/) (2019)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [Improving Semantic Segmentation via Video Propagation and Label Relaxation](https://mlsanity.com/paper/1812.01593/) (2018)
- [Disentangled Non-Local Neural Networks](https://mlsanity.com/paper/2006.06668/) (2020)
- [Context Prior for Scene Segmentation](https://mlsanity.com/paper/2004.01547/) (2020)
- [Strip Pooling: Rethinking Spatial Pooling for Scene Parsing](https://mlsanity.com/paper/2003.13328/) (2020)
- [Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing](https://mlsanity.com/paper/1810.08705/) (2018)
- [All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentation](https://mlsanity.com/paper/1903.12212/) (2019)
- [Multi-Task Self-Training for Learning General Representations](https://mlsanity.com/paper/2108.11353/) (2021)
- [ACFNet: Attentional Class Feature Network for Semantic Segmentation](https://mlsanity.com/paper/1909.09408/) (2019)

## Built on by (most cited first, 11 of 11)

- [Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptation](https://mlsanity.com/paper/2312.03502/) (2023)
- [CL-MAE: Curriculum-Learned Masked Autoencoders](https://mlsanity.com/paper/2308.16572/) (2023)
- [Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets](https://mlsanity.com/paper/2408.12489/) (2024)
- [Global High Categorical Resolution Land Cover Mapping via Weak Supervision](https://mlsanity.com/paper/2406.00891/) (2024)
- [GenGMM: Generalized Gaussian-Mixture-based Domain Adaptation Model for Semantic Segmentation](https://mlsanity.com/paper/2410.16485/) (2024)
- [Knowledge Distillation for Semantic Segmentation: A Label Space Unification Approach](https://mlsanity.com/paper/2502.19177/) (2025)
- [COARSE: Collaborative Pseudo-Labeling with Coarse Real Labels for Off-Road Semantic Segmentation](https://mlsanity.com/paper/2503.03947/) (2025)
- [Multi-modal video data-pipelines for machine learning with minimal human supervision](https://mlsanity.com/paper/2510.14862/) (2025)
- [Rewis3d: Reconstruction Improves Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2603.06374/) (2026)
- [Do Instance Priors Help Weakly Supervised Semantic Segmentation?](https://mlsanity.com/paper/2604.11170/) (2026)
- [Coarse-to-Fine Domain Incremental Learning with Attentive Distillation for Mining Footprint Segmentation in Multispectral Imagery](https://mlsanity.com/paper/2605.24460/) (2026)

## Abstract (arXiv)

For best performance, today's semantic segmentation methods use large and carefully labeled datasets, requiring expensive annotation budgets. In this work, we show that coarse annotation is a low-cost but highly effective alternative for training semantic segmentation models. Considering the urban scene segmentation scenario, we leverage cheap coarse annotations for real-world captured data, as well as synthetic data to train our model and show competitive performance compared with finely annotated real-world data. Specifically, we propose a coarse-to-fine self-training framework that generates pseudo labels for unlabeled regions of the coarsely annotated data, using synthetic data to improve predictions around the boundaries between semantic classes, and using cross-domain data augmentation to increase diversity. Our extensive experimental results on Cityscapes and BDD100k datasets demonstrate that our method achieves a significantly better performance vs annotation cost tradeoff, yielding a comparable performance to fully annotated data with only a small fraction of the annotation budget. Also, when used as pretraining, our framework performs better compared to the standard fully supervised setting.