mlsanityData to 7 Oct 2026

Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation

arXiv 2407.12449 ↗First posted 17 Jul 2024cs.CV cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Despite the substantial progress in deep learning, its adoption in industrial robotics projects remains limited, primarily due to challenges in data acquisition and labeling. Previous sim2real approaches using domain randomization require extensive scene and model optimization. To address these issues, we introduce an innovative physically-based structured light simulation system, generating both RGB and physically realistic depth images, surpassing previous dataset generation tools. We create an RGBD dataset tailored for robotic industrial grasping scenarios and evaluate it across various tasks, including object detection, instance segmentation, and embedding sim2real visual perception in industrial robotic grasping. By reducing the sim2real gap and enhancing deep learning training, we facilitate the application of deep learning models in industrial settings. Project details are available at https://baikaixinpublic.github.io/structured light 3D synthesizer/.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 14 of its 35 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

14 of 35 references.

References · 14

  1. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  2. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  3. 2016Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networkscited by 307
  4. 2020Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentationcited by 64
  5. 2017LabelFusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenescited by 32
  6. 2021NViSII: A Scriptable Tool for Photorealistic Image Generationcited by 24
  7. 2017DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognitioncited by 15
  8. 2020BlendTorch: A Real-Time, Adaptive Domain Randomization Librarycited by 10
  9. 2018Keep it Unreal: Bridging the Realism Gap for 2.5D Recognition with Geometry Priors Onlycited by 9
  10. 2022Cluttered Food Grasping with Adaptive Fingers and Synthetic-Data Trained Object Detectioncited by 8
  11. 2018Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Datacited by 7
  12. 2019Synthetic dataset generation for object-to-model deep learning in industrial applicationscited by 6
2 more
  1. 2022StandardSim: A Synthetic Dataset For Retail Environmentscited by 6
  2. 2022Sim-to-Real Transfer of Robotic Assembly with Visual Inputs Using CycleGAN and Force Controlcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025MetaSym: A Symplectic Meta-learning Framework for Physical Intelligencecited by 5
  2. 2025FUNCanon: Learning Pose-Aware Action Primitives via Functional Object Canonicalization for Generalizable Robotic Manipulationcited by 3
  3. 2025Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusioncited by 2
  4. 2025VIRTUS-FPP: Virtual Sensor Modeling for Fringe Projection Profilometry in NVIDIA Isaac Simcited by 2
  5. 2026Ontology-Guided Diffusion for Zero-Shot Visual Sim2Real Transfercited by 1
  6. 2025A Synthetic Data Pipeline for Supporting Manufacturing SMEs in Visual Assembly Controlcited by 0