# Effective Use of Synthetic Data for Urban Scene Semantic Segmentation

arXiv: https://arxiv.org/abs/1807.06132 · HTML: https://mlsanity.com/paper/1807.06132/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2018.

## Built on by (most cited first, 30 of 41)

- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [FSDR: Frequency Space Domain Randomization for Domain Generalization](https://mlsanity.com/paper/2103.02370/) (2021)
- [Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data](https://mlsanity.com/paper/2110.03374/) (2021)
- [PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training](https://mlsanity.com/paper/2105.08128/) (2021)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Category Contrast for Unsupervised Domain Adaptation in Visual Tasks](https://mlsanity.com/paper/2106.02885/) (2021)
- [RDA: Robust Domain Adaptation via Fourier Adversarial Attacking](https://mlsanity.com/paper/2106.02874/) (2021)
- [Synthetic Datasets for Autonomous Driving: A Survey](https://mlsanity.com/paper/2304.12205/) (2023)
- [Cross-View Regularization for Domain Adaptive Panoptic Segmentation](https://mlsanity.com/paper/2103.02584/) (2021)
- [Deep Symmetric Adaptation Network for Cross-modality Medical Image Segmentation](https://mlsanity.com/paper/2101.06853/) (2021)
- [Feature-based Style Randomization for Domain Generalization](https://mlsanity.com/paper/2106.03171/) (2021)
- [Cross-View Cross-Scene Multi-View Crowd Counting](https://mlsanity.com/paper/2205.01551/) (2022)
- [Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection](https://mlsanity.com/paper/2103.14198/) (2021)
- [Compositional Generative Model of Unbounded 4D Cities](https://mlsanity.com/paper/2501.08983/) (2025)
- [Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation](https://mlsanity.com/paper/2309.09637/) (2023)
- [Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Driving](https://mlsanity.com/paper/2309.04422/) (2023)
- [SPARK: SPAcecraft Recognition leveraging Knowledge of Space Environment](https://mlsanity.com/paper/2104.05978/) (2021)
- [Unsupervised Adaptation of Semantic Segmentation Models without Source Data](https://mlsanity.com/paper/2112.02359/) (2021)
- [Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning](https://mlsanity.com/paper/2203.10395/) (2022)
- [SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets](https://mlsanity.com/paper/2308.11880/) (2023)
- [A Procedural World Generation Framework for Systematic Evaluation of Continual Learning](https://mlsanity.com/paper/2106.02585/) (2021)
- [Semi-Supervised Domain Adaptation via Adaptive and Progressive Feature Alignment](https://mlsanity.com/paper/2106.02845/) (2021)
- [Trans2k: Unlocking the Power of Deep Models for Transparent Object Tracking](https://mlsanity.com/paper/2210.03436/) (2022)
- [A New Dataset and a Distractor-Aware Architecture for Transparent Object Tracking](https://mlsanity.com/paper/2401.03872/) (2024)
- [A Bayesian Approach to Reinforcement Learning of Vision-Based Vehicular Control](https://mlsanity.com/paper/2104.03807/) (2021)
- [DRIV100: In-The-Wild Multi-Domain Dataset and Evaluation for Real-World Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/2102.00150/) (2021)
- [Validation of object detection in UAV-based images using synthetic data](https://mlsanity.com/paper/2201.06629/) (2022)
- [DropTrack -- automatic droplet tracking using deep learning for microfluidic applications](https://mlsanity.com/paper/2205.02568/) (2022)
- [Domain-invariant Prototypes for Semantic Segmentation](https://mlsanity.com/paper/2208.06087/) (2022)

## Abstract (arXiv)

Training a deep network to perform semantic segmentation requires large amounts of labeled data. To alleviate the manual effort of annotating real images, researchers have investigated the use of synthetic data, which can be labeled automatically. Unfortunately, a network trained on synthetic data performs relatively poorly on real images. While this can be addressed by domain adaptation, existing methods all require having access to real images during training. In this paper, we introduce a drastically different way to handle synthetic images that does not require seeing any real images at training time. Our approach builds on the observation that foreground and background classes are not affected in the same manner by the domain shift, and thus should be treated differently. In particular, the former should be handled in a detection-based manner to better account for the fact that, while their texture in synthetic images is not photo-realistic, their shape looks natural. Our experiments evidence the effectiveness of our approach on Cityscapes and CamVid with models trained on synthetic data only.