# Object Detection Using Sim2Real Domain Randomization for Robotic Applications

arXiv: https://arxiv.org/abs/2208.04171 · HTML: https://mlsanity.com/paper/2208.04171/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 87 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [R-FCN: Object Detection via Region-based Fully Convolutional Networks](https://mlsanity.com/paper/1605.06409/) (2016)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?](https://mlsanity.com/paper/1610.01983/) (2016)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)

## Built on by (most cited first, 25 of 25)

- [Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer](https://mlsanity.com/paper/2302.14332/) (2023)
- [ImaginaryNet: Learning Object Detectors without Real Images and Annotations](https://mlsanity.com/paper/2210.06886/) (2022)
- [DGNR: Density-Guided Neural Point Rendering of Large Driving Scenes](https://mlsanity.com/paper/2311.16664/) (2023)
- [World Models for General Surgical Grasping](https://mlsanity.com/paper/2405.17940/) (2024)
- [Towards Sim-to-Real Industrial Parts Classification with Synthetic Dataset](https://mlsanity.com/paper/2404.08778/) (2024)
- [Domain Randomization for Object Detection in Manufacturing Applications using Synthetic Data: A Comprehensive Study](https://mlsanity.com/paper/2506.07539/) (2025)
- [Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control](https://mlsanity.com/paper/2302.11694/) (2023)
- [Sim-to-Real Grasp Detection with Global-to-Local RGB-D Adaptation](https://mlsanity.com/paper/2403.11511/) (2024)
- [HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments](https://mlsanity.com/paper/2304.05826/) (2023)
- [Synthetic Industrial Object Detection: GenAI vs. Feature-Based Methods](https://mlsanity.com/paper/2511.23241/) (2025)
- [Fully-Synthetic Training for Visual Quality Inspection in Automotive Production](https://mlsanity.com/paper/2503.09354/) (2025)
- [Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Tracking](https://mlsanity.com/paper/2410.14685/) (2024)
- [Synthetic Similarity Search in Automotive Production](https://mlsanity.com/paper/2505.07256/) (2025)
- [Visuomotor Grasping with World Models for Surgical Robots](https://mlsanity.com/paper/2508.11200/) (2025)
- [FOD-S2R: A FOD Dataset for Sim2Real Transfer Learning based Object Detection](https://mlsanity.com/paper/2512.01315/) (2025)
- [SynthRender and I-AsSET: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perception](https://mlsanity.com/paper/2602.21141/) (2026)
- [CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environment](https://mlsanity.com/paper/2604.05484/) (2026)
- [Auditing Metrics for Synthetic Data Utility: A Cross-Domain Study of Appearance and Geometry](https://mlsanity.com/paper/2605.22467/) (2026)
- [Position: Deployed Reinforcement Learning should be Continual](https://mlsanity.com/paper/2606.04029/) (2026)
- [SuDA: Support-based Domain Adaptation for Sim2Real Motion Capture with Flexible Sensors](https://mlsanity.com/paper/2405.16152/) (2024)
- [Surveying You Only Look Once (YOLO) Multispectral Object Detection Advancements, Applications And Challenges](https://mlsanity.com/paper/2409.12977/) (2024)
- [Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Pooling](https://mlsanity.com/paper/2505.18446/) (2025)
- [GaussFly: Contrastive Reinforcement Learning for Visuomotor Policies in 3D Gaussian Fields](https://mlsanity.com/paper/2604.05062/) (2026)
- [Removing Infrastructure Barriers in Human-Robot Collaboration Through Wireless Reconfigurable Cells](https://mlsanity.com/paper/2608.09658/) (2026)
- [Adapting Vision-Language Models for Human-Readable XAI in Industrial Object Detection](https://mlsanity.com/paper/2609.31690/) (2026)

## Abstract (arXiv)

Robots working in unstructured environments must be capable of sensing and interpreting their surroundings. One of the main obstacles of deep-learning-based models in the field of robotics is the lack of domain-specific labeled data for different industrial applications. In this article, we propose a sim2real transfer learning method based on domain randomization for object detection with which labeled synthetic datasets of arbitrary size and object types can be automatically generated. Subsequently, a state-of-the-art convolutional neural network, YOLOv4, is trained to detect the different types of industrial objects. With the proposed domain randomization method, we could shrink the reality gap to a satisfactory level, achieving 86.32% and 97.38% mAP50 scores, respectively, in the case of zero-shot and one-shot transfers, on our manually annotated dataset containing 190 real images. Our solution fits for industrial use as the data generation process takes less than 0.5 s per image and the training lasts only around 12 h, on a GeForce RTX 2080 Ti GPU. Furthermore, it can reliably differentiate similar classes of objects by having access to only one real image for training. To our best knowledge, this is the only work thus far satisfying these constraints.