# Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data

arXiv: https://arxiv.org/abs/1909.00889 · HTML: https://mlsanity.com/paper/1909.00889/

## Key facts

- Cited by 246 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2019.

## Built on by (most cited first, 30 of 246)

- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space](https://mlsanity.com/paper/2103.06030/) (2021)
- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Improving Out-of-Distribution Robustness via Selective Augmentation](https://mlsanity.com/paper/2201.00299/) (2022)
- [FSDR: Frequency Space Domain Randomization for Domain Generalization](https://mlsanity.com/paper/2103.02370/) (2021)
- [Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization](https://mlsanity.com/paper/2110.10832/) (2021)
- [Flow as the Cross-Domain Manipulation Interface](https://mlsanity.com/paper/2407.15208/) (2024)
- [Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization](https://mlsanity.com/paper/2203.07740/) (2022)
- [Model-Based Domain Generalization](https://mlsanity.com/paper/2102.11436/) (2021)
- [Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time](https://mlsanity.com/paper/2211.14238/) (2022)
- [Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2204.02548/) (2022)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [NICO++: Towards Better Benchmarking for Domain Generalization](https://mlsanity.com/paper/2204.08040/) (2022)
- [Sparse Mixture-of-Experts are Domain Generalizable Learners](https://mlsanity.com/paper/2206.04046/) (2022)
- [Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.16765/) (2021)
- [Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2312.04265/) (2023)
- [Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.11249/) (2021)
- [Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusion](https://mlsanity.com/paper/2302.03298/) (2023)
- [Semantic-Aware Domain Generalized Segmentation](https://mlsanity.com/paper/2204.00822/) (2022)
- [WildNet: Learning Domain Generalized Semantic Segmentation from the Wild](https://mlsanity.com/paper/2204.01446/) (2022)
- [Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation](https://mlsanity.com/paper/2207.04892/) (2022)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentation](https://mlsanity.com/paper/2108.02376/) (2021)
- [C-Mixup: Improving Generalization in Regression](https://mlsanity.com/paper/2210.05775/) (2022)
- [Label-Free Liver Tumor Segmentation](https://mlsanity.com/paper/2303.14869/) (2023)
- [DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations](https://mlsanity.com/paper/2201.09637/) (2022)
- [Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects](https://mlsanity.com/paper/2208.03792/) (2022)
- [Pin the Memory: Learning to Generalize Semantic Segmentation](https://mlsanity.com/paper/2204.03609/) (2022)

## Abstract (arXiv)

We propose to harness the potential of simulation for the semantic segmentation of real-world self-driving scenes in a domain generalization fashion. The segmentation network is trained without any data of target domains and tested on the unseen target domains. To this end, we propose a new approach of domain randomization and pyramid consistency to learn a model with high generalizability. First, we propose to randomize the synthetic images with the styles of real images in terms of visual appearances using auxiliary datasets, in order to effectively learn domain-invariant representations. Second, we further enforce pyramid consistency across different "stylized" images and within an image, in order to learn domain-invariant and scale-invariant features, respectively. Extensive experiments are conducted on the generalization from GTA and SYNTHIA to Cityscapes, BDDS and Mapillary; and our method achieves superior results over the state-of-the-art techniques. Remarkably, our generalization results are on par with or even better than those obtained by state-of-the-art simulation-to-real domain adaptation methods, which access the target domain data at training time.