Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
Not on the map. Why?
Abstract (arXiv)
We present structured domain randomization (SDR), a variant of domain randomization (DR) that takes into account the structure and context of the scene. In contrast to DR, which places objects and distractors randomly according to a uniform probability distribution, SDR places objects and distractors randomly according to probability distributions that arise from the specific problem at hand. In this manner, SDR-generated imagery enables the neural network to take the context around an object into consideration during detection. We demonstrate the power of SDR for the problem of 2D bounding box car detection, achieving competitive results on real data after training only on synthetic data. On the KITTI easy, moderate, and hard tasks, we show that SDR outperforms other approaches to generating synthetic data (VKITTI, Sim 200k, or DR), as well as real data collected in a different domain (BDD100K). Moreover, synthetic SDR data combined with real KITTI data outperforms real KITTI data alone.
Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 21; 2022: 19; 2023: 24; 2024: 16; 2025: 11; 2026: 3.
94 papers.
Sort:
By area:
- 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
- 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
- 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
- 2022Parameter-free Online Test-time Adaptationcited by 184
- 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 163
- 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
- 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
- 2023Language Conditioned Traffic Generationcited by 86
- 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
- 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
- 2022MIME: Human-Aware 3D Scene Generationcited by 57
- 2022Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objectscited by 49
38 more
- 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
- 2023All for One, and One for All: UrbanSyn Dataset, the third Musketeer of Synthetic Driving Scenescited by 40
- 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
- 2021Feature-based Style Randomization for Domain Generalizationcited by 34
- 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
- 2022TTTFlow: Unsupervised Test-Time Training with Normalizing Flowcited by 33
- 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 30
- 2022Attention Diversification for Domain Generalizationcited by 29
- 2021Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Datacited by 26
- 2021NViSII: A Scriptable Tool for Photorealistic Image Generationcited by 24
- 2021Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehiclescited by 22
- 2023ClusT3: Information Invariant Test-Time Trainingcited by 21
- 2021CausalAF: Causal Autoregressive Flow for Safety-Critical Driving Scenario Generationcited by 20
- 2023SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervisioncited by 20
- 2023SurvivalGAN: Generating Time-to-Event Data for Survival Analysiscited by 17
- 2021Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehiclescited by 16
- 2023CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embeddingcited by 16
- 2022PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Datacited by 14
- 2023Model Barrier: A Compact Un-Transferable Isolation Domain for Model Intellectual Property Protectioncited by 14
- 2021Towards Optimal Strategies for Training Self-Driving Perception Models in Simulationcited by 12
- 2021PeopleSansPeople: A Synthetic Data Generator for Human-Centric Computer Visioncited by 11
- 2022Heterogeneous Trajectory Forecasting via Risk and Scene Graph Learningcited by 11
- 2023A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptationcited by 11
- 2023Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweightingcited by 9
- 2025Enhancing Autonomous Driving Safety with Collision Scenario Integrationcited by 9
- 2022SimVQA: Exploring Simulated Environments for Visual Question Answeringcited by 8
- 2023Aggregation of Disentanglement: Reconsidering Domain Variations in Domain Generalizationcited by 8
- 2023Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Levelcited by 7
- 2024Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wildcited by 7
- 2024Towards Sim-to-Real Industrial Parts Classification with Synthetic Datasetcited by 7
- 2024Say No to Freeloader: Protecting Intellectual Property of Your Deep Modelcited by 7
- 2024Improving Object Detector Training on Synthetic Data by Starting With a Strong Baseline Methodologycited by 6
- 2024Bridging the Domain Gap for Flight-Ready Spaceborne Visioncited by 6
- 2025Domain Randomization for Object Detection in Manufacturing Applications using Synthetic Data: A Comprehensive Studycited by 6
- 2021Synthetic training data generation for deep learning based quality inspectioncited by 5
- 2022Photo-realistic Neural Domain Randomizationcited by 5
- 2023Learning Human Action Recognition Representations Without Real Humanscited by 5
- 2024Phrase Grounding-based Style Transfer for Single-Domain Generalized Object Detectioncited by 5
50 most cited of 94 All 94 (JSON)
- 2026Semantically-Guided Domain Randomization for Industrial Object Detection in Low-Image-Budget Regimes22 Sep 2026
- 2026Diffuse the object, keep its label: curating detector training data from a few unlabeled photographs via VLM-built 3D vegetation scenes10 Aug 2026
- 2026Adversarial Batch Representation Augmentation for Batch Correction in High-Content Cellular Screening5 Mar 2026
- 2025Scaling Up Forest Vision with Synthetic Data14 Sep 2025
- 2025Class Unbiasing for Generalization in Medical Diagnosis9 Aug 2025
- 2025Set Pivot Learning: Redefining Generalized Segmentation with Vision Foundation Models3 Aug 2025
- 2025Improving Generalization Ability of Robotic Imitation Learning by Resolving Causal Confusion in Observations30 Jul 2025
- 2025Synthetic Data Matters: Re-training with Geo-typical Synthetic Labels for Building Detection22 Jul 2025
- 2025Domain Randomization for Object Detection in Manufacturing Applications using Synthetic Data: A Comprehensive Study9 Jun 2025
- 2025LangDAug: Langevin Data Augmentation for Multi-Source Domain Generalization in Medical Image Segmentation26 May 2025
- 2025Artificial Conversations, Real Results: Fostering Language Detection with Synthetic Data31 Mar 2025
- 2025Automating 3D Dataset Generation with Neural Radiance Fields20 Mar 2025
38 more
- 2025Enhancing Autonomous Driving Safety with Collision Scenario Integration5 Mar 2025
- 2025Invariance Pair Guidance: Robustness to Spurious Correlations via Corrective Gradients26 Feb 2025
- 2024Drone Detection using Deep Neural Networks Trained on Pure Synthetic Data13 Nov 2024
- 2024Bridging the Domain Gap for Flight-Ready Spaceborne Vision18 Sep 2024
- 2024Say No to Freeloader: Protecting Intellectual Property of Your Deep Model23 Aug 2024
- 2024Weak-to-Strong Compositional Learning from Generative Models for Language-based Object Detection21 Jul 2024
- 2024Solving Motion Planning Tasks with a Scalable Generative Model3 Jul 2024
- 2024Mixed Diffusion for 3D Indoor Scene Synthesis31 May 2024
- 2024Improving Object Detector Training on Synthetic Data by Starting With a Strong Baseline Methodology30 May 2024
- 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles22 May 2024
- 2024Augmenting Safety-Critical Driving Scenarios while Preserving Similarity to Expert Trajectories20 Apr 2024
- 2024Towards Sim-to-Real Industrial Parts Classification with Synthetic Dataset12 Apr 2024
- 2024NC-TTT: A Noise Contrastive Approach for Test-Time Training12 Apr 2024
- 2024Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wild21 Mar 2024
- 2024Investigation of the Impact of Synthetic Training Data in the Industrial Application of Terminal Strip Object Detection6 Mar 2024
- 2024Learning Conditional Invariances through Non-Commutativity18 Feb 2024
- 2024Phrase Grounding-based Style Transfer for Single-Domain Generalized Object Detection2 Feb 2024
- 2024Determination of efficiency indicators of the stand for intelligent control of manual operations in industrial production19 Jan 2024
- 2023All for One, and One for All: UrbanSyn Dataset, the third Musketeer of Synthetic Driving Scenes19 Dec 2023
- 2023Synthetic Data Generation for Bridging Sim2Real Gap in a Production Environment18 Nov 2023
- 2023NormNet: Scale Normalization for 6D Pose Estimation in Stacked Scenarios15 Nov 2023
- 2023Learning Human Action Recognition Representations Without Real Humans10 Nov 2023
- 2023ClusT3: Information Invariant Test-Time Training18 Oct 2023
- 2023Alice Benchmarks: Connecting Real World Re-Identification with the Synthetic6 Oct 2023
- 2023Prompting-based Temporal Domain Generalization3 Oct 2023
- 2023Towards Generalising Neural Topical Representations24 Jul 2023
- 2023Language Conditioned Traffic Generation16 Jul 2023
- 2023Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorization8 Jun 2023
- 2023Realistically distributing object placements in synthetic training data improves the performance of vision-based object detection models24 May 2023
- 2023Synthetic Datasets for Autonomous Driving: A Survey24 Apr 2023
- 2023SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervision30 Mar 2023
- 2023Model Barrier: A Compact Un-Transferable Isolation Domain for Model Intellectual Property Protection20 Mar 2023
- 2023A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation16 Mar 2023
- 2023HazardNet: Road Debris Detection by Augmentation of Synthetic Models14 Mar 2023
- 2023CLIP-Layout: Style-Consistent Indoor Scene Synthesis with Semantic Furniture Embedding7 Mar 2023
- 2023Adaptive Texture Filtering for Single-Domain Generalized Segmentation6 Mar 2023
- 2023Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level3 Mar 2023
- 2023SurvivalGAN: Generating Time-to-Event Data for Survival Analysis24 Feb 2023
50 newest of 94 All 94 (JSON)
- 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
- 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
- 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
- 2022Parameter-free Online Test-time Adaptationcited by 184
- 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 163
- 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
- 2023Language Conditioned Traffic Generationcited by 86
- 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 83
- 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
- 2022MIME: Human-Aware 3D Scene Generationcited by 57
- 2022Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objectscited by 49
- 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
12 most cited of 85 All 85 (JSON)
- 2026Diffuse the object, keep its label: curating detector training data from a few unlabeled photographs via VLM-built 3D vegetation scenes10 Aug 2026
- 2026Adversarial Batch Representation Augmentation for Batch Correction in High-Content Cellular Screening5 Mar 2026
- 2025Scaling Up Forest Vision with Synthetic Data14 Sep 2025
- 2025Class Unbiasing for Generalization in Medical Diagnosis9 Aug 2025
- 2025Set Pivot Learning: Redefining Generalized Segmentation with Vision Foundation Models3 Aug 2025
- 2025Synthetic Data Matters: Re-training with Geo-typical Synthetic Labels for Building Detection22 Jul 2025
- 2025Domain Randomization for Object Detection in Manufacturing Applications using Synthetic Data: A Comprehensive Study9 Jun 2025
- 2025LangDAug: Langevin Data Augmentation for Multi-Source Domain Generalization in Medical Image Segmentation26 May 2025
- 2025Automating 3D Dataset Generation with Neural Radiance Fields20 Mar 2025
- 2025Invariance Pair Guidance: Robustness to Spurious Correlations via Corrective Gradients26 Feb 2025
- 2024Drone Detection using Deep Neural Networks Trained on Pure Synthetic Data13 Nov 2024
- 2024Bridging the Domain Gap for Flight-Ready Spaceborne Vision18 Sep 2024
12 newest of 85 All 85 (JSON)