mlsanityData to 13 Oct 2026

Object Detection Using Sim2Real Domain Randomization for Robotic Applications

arXiv 2208.04171 ↗First posted 8 Aug 2022cs.RO cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

Robots working in unstructured environments must be capable of sensing and interpreting their surroundings. One of the main obstacles of deep-learning-based models in the field of robotics is the lack of domain-specific labeled data for different industrial applications. In this article, we propose a sim2real transfer learning method based on domain randomization for object detection with which labeled synthetic datasets of arbitrary size and object types can be automatically generated. Subsequently, a state-of-the-art convolutional neural network, YOLOv4, is trained to detect the different types of industrial objects. With the proposed domain randomization method, we could shrink the reality gap to a satisfactory level, achieving 86.32% and 97.38% mAP50 scores, respectively, in the case of zero-shot and one-shot transfers, on our manually annotated dataset containing 190 real images. Our solution fits for industrial use as the data generation process takes less than 0.5 s per image and the training lasts only around 12 h, on a GeForce RTX 2080 Ti GPU. Furthermore, it can reliably differentiate similar classes of objects by having access to only one real image for training. To our best knowledge, this is the only work thus far satisfying these constraints.

Key facts. Cited by 24 papers in the map (data through 13 Oct 2026). Builds on 40 of its 87 references that are in the map. First posted on arXiv 8 Aug 2022.

Builds on

40 of 87 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,321
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,625
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
  7. 2022Generative Adversarial Networkscited by 5,614
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,027
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,840
  10. 2013Auto-Encoding Variational Bayescited by 4,024
  11. 2015SSD: Single Shot MultiBox Detectorcited by 3,128
  12. 2015Fast R-CNNcited by 2,933
28 more
  1. 2018YOLOv3: An Incremental Improvementcited by 2,655
  2. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  3. 2013Describing Textures in the Wildcited by 2,115
  4. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,628
  5. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,627
  6. 2016YOLO9000: Better, Faster, Strongercited by 1,430
  7. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,411
  8. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  9. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 720
  10. 2019Object Detection in 20 Years: A Surveycited by 528
  11. 2018Domain Adaptive Faster R-CNN for Object Detection in the Wildcited by 442
  12. 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
  13. 2017Semantic Foggy Scene Understanding with Synthetic Datacited by 401
  14. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 341
  15. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  16. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 292
  17. 2016Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?cited by 194
  18. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 138
  19. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 131
  20. 2017Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Taskcited by 66
  21. 2017On Pre-Trained Image Features and Synthetic Images for Deep Learningcited by 51
  22. 2017Domain Randomization and Generative Models for Robotic Graspingcited by 34
  23. 2020Bodies at Rest: 3D Human Pose and Shape Estimation from a Pressure Image using Synthetic Datacited by 34
  24. 2021BodyPressure -- Inferring Body Pose and Contact Pressure from a Depth Imagecited by 20
  25. 2019Learning to Augment Synthetic Images for Sim2Real Policy Transfercited by 16
  26. 2021Generation of GelSight Tactile Images for Sim2Real Learningcited by 14
  27. 2017Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policiescited by 11
  28. 2020Simultaneously-Collected Multimodal Lying Pose Dataset: Towards In-Bed Human Pose Monitoring under Adverse Vision Conditionscited by 9

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 6; 2025: 7; 2026: 6.

24 papers.

Sort:

By area:

  1. 2023Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfercited by 24
  2. 2022ImaginaryNet: Learning Object Detectors without Real Images and Annotationscited by 16
  3. 2023DGNR: Density-Guided Neural Point Rendering of Large Driving Scenescited by 9
  4. 2024World Models for General Surgical Graspingcited by 9
  5. 2024Towards Sim-to-Real Industrial Parts Classification with Synthetic Datasetcited by 7
  6. 2025Domain Randomization for Object Detection in Manufacturing Applications using Synthetic Data: A Comprehensive Studycited by 6
  7. 2023Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Controlcited by 5
  8. 2024Sim-to-Real Grasp Detection with Global-to-Local RGB-D Adaptationcited by 4
  9. 2023HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environmentscited by 3
  10. 2025Synthetic Industrial Object Detection: GenAI vs. Feature-Based Methodscited by 3
  11. 2025Fully-Synthetic Training for Visual Quality Inspection in Automotive Productioncited by 2
  12. 2024Leveraging Event Streams with Deep Reinforcement Learning for End-to-End UAV Trackingcited by 1
12 more
  1. 2025Synthetic Similarity Search in Automotive Productioncited by 1
  2. 2025Visuomotor Grasping with World Models for Surgical Robotscited by 1
  3. 2025FOD-S2R: A FOD Dataset for Sim2Real Transfer Learning based Object Detectioncited by 1
  4. 2026SynthRender and I-AsSET: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perceptioncited by 1
  5. 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environmentcited by 1
  6. 2026Auditing Metrics for Synthetic Data Utility: A Cross-Domain Study of Appearance and Geometrycited by 1
  7. 2026Position: Deployed Reinforcement Learning should be Continualcited by 1
  8. 2024SuDA: Support-based Domain Adaptation for Sim2Real Motion Capture with Flexible Sensorscited by 0
  9. 2024Surveying You Only Look Once (YOLO) Multispectral Object Detection Advancements, Applications And Challengescited by 0
  10. 2025Mitigating Context Bias in Domain Adaptation for Object Detection using Mask Poolingcited by 0
  11. 2026Removing Infrastructure Barriers in Human-Robot Collaboration Through Wireless Reconfigurable Cellscited by 0
  12. 2026Adapting Vision-Language Models for Human-Readable XAI in Industrial Object Detectioncited by 0