# Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data

arXiv: https://arxiv.org/abs/1810.10093 · HTML: https://mlsanity.com/paper/1810.10093/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2018.

## Built on by (most cited first, 30 of 94)

- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [Parameter-free Online Test-time Adaptation](https://mlsanity.com/paper/2201.05718/) (2022)
- [FSDR: Frequency Space Domain Randomization for Domain Generalization](https://mlsanity.com/paper/2103.02370/) (2021)
- [ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles](https://mlsanity.com/paper/2405.14062/) (2024)
- [Language Conditioned Traffic Generation](https://mlsanity.com/paper/2307.07947/) (2023)
- [NICO++: Towards Better Benchmarking for Domain Generalization](https://mlsanity.com/paper/2204.08040/) (2022)
- [SceneGen: Learning to Generate Realistic Traffic Scenes](https://mlsanity.com/paper/2101.06541/) (2021)
- [Multi-Modal 3D Object Detection in Autonomous Driving: a Survey](https://mlsanity.com/paper/2106.12735/) (2021)
- [MIME: Human-Aware 3D Scene Generation](https://mlsanity.com/paper/2212.04360/) (2022)
- [Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects](https://mlsanity.com/paper/2208.03792/) (2022)
- [Synthetic Datasets for Autonomous Driving: A Survey](https://mlsanity.com/paper/2304.12205/) (2023)
- [All for One, and One for All: UrbanSyn Dataset, the third Musketeer of Synthetic Driving Scenes](https://mlsanity.com/paper/2312.12176/) (2023)
- [Solving Motion Planning Tasks with a Scalable Generative Model](https://mlsanity.com/paper/2407.02797/) (2024)
- [Feature-based Style Randomization for Domain Generalization](https://mlsanity.com/paper/2106.03171/) (2021)
- [Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning](https://mlsanity.com/paper/2207.09081/) (2022)
- [TTTFlow: Unsupervised Test-Time Training with Normalizing Flow](https://mlsanity.com/paper/2210.11389/) (2022)
- [Mixed Diffusion for 3D Indoor Scene Synthesis](https://mlsanity.com/paper/2405.21066/) (2024)
- [Attention Diversification for Domain Generalization](https://mlsanity.com/paper/2210.04206/) (2022)
- [Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Data](https://mlsanity.com/paper/2112.00054/) (2021)
- [NViSII: A Scriptable Tool for Photorealistic Image Generation](https://mlsanity.com/paper/2105.13962/) (2021)
- [Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles](https://mlsanity.com/paper/2112.12252/) (2021)
- [SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervision](https://mlsanity.com/paper/2303.17200/) (2023)
- [ClusT3: Information Invariant Test-Time Training](https://mlsanity.com/paper/2310.12345/) (2023)
- [CausalAF: Causal Autoregressive Flow for Safety-Critical Driving Scenario Generation](https://mlsanity.com/paper/2110.13939/) (2021)
- [SurvivalGAN: Generating Time-to-Event Data for Survival Analysis](https://mlsanity.com/paper/2302.12749/) (2023)
- [Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles](https://mlsanity.com/paper/2103.14098/) (2021)
- [CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding](https://mlsanity.com/paper/2303.03565/) (2023)
- [PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Data](https://mlsanity.com/paper/2212.04821/) (2022)

## Abstract (arXiv)

We present structured domain randomization (SDR), a variant of domain randomization (DR) that takes into account the structure and context of the scene. In contrast to DR, which places objects and distractors randomly according to a uniform probability distribution, SDR places objects and distractors randomly according to probability distributions that arise from the specific problem at hand. In this manner, SDR-generated imagery enables the neural network to take the context around an object into consideration during detection. We demonstrate the power of SDR for the problem of 2D bounding box car detection, achieving competitive results on real data after training only on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that SDR outperforms other approaches to generating synthetic data (VKITTI, Sim 200k, or DR), as well as real data collected in a different domain (BDD100K). Moreover, synthetic SDR data combined with real KITTI data outperforms real KITTI data alone.