mlsanityData to 14 Oct 2026

Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization

arXiv 1804.06516 ↗First posted 18 Apr 2018cs.CVcited by 294

Not on the map. Why?

Abstract (arXiv)

We present a system for training deep neural networks for object detection using synthetic images. To handle the variability in real-world data, the system relies upon the technique of domain randomization, in which the parameters of the simulator-such as lighting, pose, object textures, etc.-are randomized in non-realistic ways to force the neural network to learn the essential features of the object of interest. We explore the importance of these parameters, showing that it is possible to produce a network with compelling performance using only non-artistically-generated synthetic data. With additional fine-tuning on real data, the network yields better performance than using real data alone. This result opens up the possibility of using inexpensive synthetic data for training neural networks while avoiding the need to collect large amounts of hand-annotated real-world data or to generate high-fidelity synthetic worlds-both of which remain bottlenecks for many applications. The approach is evaluated on bounding box detection of cars on the KITTI dataset.

Key facts. Cited by 294 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 53; 2022: 53; 2023: 53; 2024: 50; 2025: 41; 2026: 44.

294 papers.

Sort:

By area:

  1. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  2. 2022Continual Test-Time Domain Adaptationcited by 525
  3. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  4. 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 237
  5. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  6. 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 164
  7. 2024Flow as the Cross-Domain Manipulation Interfacecited by 132
  8. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  9. 2023DCFace: Synthetic Face Generation with Dual Condition Diffusion Modelcited by 116
  10. 2022Robust Mean Teacher for Continual and Gradual Test-Time Adaptationcited by 104
  11. 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
  12. 2021MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?cited by 69
38 more
  1. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 65
  2. 2023Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correctioncited by 65
  3. 2022Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objectscited by 50
  4. 2023Synthetic Datasets for Autonomous Driving: A Surveycited by 48
  5. 20233D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Cloudscited by 46
  6. 2022Deep Frequency Filtering for Domain Generalizationcited by 43
  7. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 41
  8. 2021Feature-based Style Randomization for Domain Generalizationcited by 34
  9. 2022RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesiscited by 33
  10. 2022Temporal Domain Generalization with Drift-Aware Dynamic Neural Networkscited by 33
  11. 2023TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimationcited by 32
  12. 2023Prompting Diffusion Representations for Cross-Domain Semantic Segmentationcited by 30
  13. 2025Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigationcited by 30
  14. 2022Domain Generalization using Pretrained Models without Fine-tuningcited by 29
  15. 2021CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Datacited by 28
  16. 2022DytanVO: Joint Refinement of Visual Odometry and Motion Segmentation in Dynamic Environmentscited by 27
  17. 2024Scaling laws for learning with real and surrogate datacited by 27
  18. 2021Improving Fractal Pre-trainingcited by 26
  19. 2021Taxonomy of Machine Learning Safety: A Survey and Primercited by 25
  20. 2022Object Detection Using Sim2Real Domain Randomization for Robotic Applicationscited by 25
  21. 2021NViSII: A Scriptable Tool for Photorealistic Image Generationcited by 24
  22. 2023Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfercited by 24
  23. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  24. 2026RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policiescited by 24
  25. 2021Image Compositing for Segmentation of Surgical Tools without Manual Annotationscited by 23
  26. 2021SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereocited by 18
  27. 2021Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehiclescited by 16
  28. 2021Sim2Air - Synthetic aerial dataset for UAV monitoringcited by 16
  29. 2022Rethinking Surgical Instrument Segmentation: A Background Image Can Be All You Needcited by 16
  30. 2022ImaginaryNet: Learning Object Detectors without Real Images and Annotationscited by 16
  31. 2021A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?cited by 15
  32. 2022SIMBAR: Single Image-Based Scene Relighting For Effective Data Augmentation For Automated Driving Vision Taskscited by 15
  33. 2023AnyStar: Domain randomized universal star-convex 3D instance segmentationcited by 15
  34. 2024Continuous Temporal Domain Generalizationcited by 15
  35. 2022Conditional Generation of Medical Time Series for Extrapolation to Underrepresented Populationscited by 14
  36. 2023CAD-Estate: Large-scale CAD Model Annotation in RGB Videoscited by 14
  37. 2022Learning-by-Novel-View-Synthesis for Full-Face Appearance-Based 3D Gaze Estimationcited by 13
  38. 2024textitTrans-LoRA: towards data-free Transferable Parameter Efficient Finetuningcited by 13

50 most cited of 294 All 294 (JSON)