mlsanityData to 7 Oct 2026

Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping

arXiv 1709.07857 ↗First posted 22 Sep 2017cs.LG cs.AI cs.CVcited by 138

Not on the map. Why?

Abstract (arXiv)

Instrumenting and collecting annotated visual grasping datasets to train modern machine learning algorithms can be extremely time-consuming and expensive. An appealing alternative is to use off-the-shelf simulators to render synthetic data for which ground-truth annotations are generated automatically. Unfortunately, models trained purely on simulated data often fail to generalize to the real world. We study how randomized simulated environments and domain adaptation methods can be extended to train a grasping system to grasp novel objects from raw monocular RGB images. We extensively evaluate our approaches with a total of more than 25,000 physical test grasps, studying a range of simulation conditions and domain adaptation methods, including a novel extension of pixel-level domain adaptation that we term the GraspGAN. We show that, by using synthetic data and domain adaptation, we are able to reduce the number of real-world samples needed to achieve a given level of performance by up to 50 times, using only randomly generated simulated objects. We also show that by using only unlabeled real-world data and our GraspGAN methodology, we obtain real-world grasping performance without any real-world labels that is similar to that achieved with 939,777 labeled real-world samples.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 1; 2021: 35; 2022: 29; 2023: 24; 2024: 13; 2025: 21; 2026: 13.

138 papers.

Sort:

By area:

  1. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  3. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  4. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  5. 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 101
  6. 2023On Bringing Robots Homecited by 101
  7. 2022Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectivescited by 98
  8. 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
  9. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 78
  10. 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
  11. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 70
  12. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 69
38 more
  1. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 64
  2. 2022Transferability in Deep Learning: A Surveycited by 57
  3. 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
  4. 2023QUAR-VLA: Vision-Language-Action Model for Quadruped Robotscited by 55
  5. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  6. 2021Scalable Scene Flow from Point Clouds in the Real Worldcited by 49
  7. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  8. 2022NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fieldscited by 41
  9. 2023Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learningcited by 36
  10. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  11. 2021Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overviewcited by 31
  12. 2022CPPF: Towards Robust Category-Level 9D Pose Estimation in the Wildcited by 28
  13. 2022A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologiescited by 28
  14. 2023Polybot: Training One Policy Across Robots While Embracing Variabilitycited by 28
  15. 2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 26
  16. 2024Addressing Misspecification in Simulation-based Inference through Data-driven Calibrationcited by 26
  17. 2021UDA-COPE: Unsupervised Domain Adaptation for Category-level Object Pose Estimationcited by 25
  18. 2026World-Gymnast: Training Robots with Reinforcement Learning in a World Modelcited by 25
  19. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  20. 2022Object Detection Using Sim2Real Domain Randomization for Robotic Applicationscited by 23
  21. 2023Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfercited by 23
  22. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 22
  23. 2022Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Pickingcited by 20
  24. 2019Scaling data-driven robotics with reward sketching and batch reinforcement learningcited by 18
  25. 2021Out-of-Domain Generalization from a Single Source: An Uncertainty Quantification Approachcited by 18
  26. 2023TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfercited by 18
  27. 2023Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challengescited by 17
  28. 2024ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environmentscited by 16
  29. 2022A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learningcited by 15
  30. 2023Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performancescited by 15
  31. 2022Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computerscited by 13
  32. 2021Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Groundingcited by 12
  33. 2022Visuo-Haptic Object Perception for Robots: An Overviewcited by 12
  34. 2021Collision-Aware Target-Driven Object Grasping in Constrained Environmentscited by 11
  35. 2021REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Cluttercited by 11
  36. 2022Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Visioncited by 11
  37. 2024Generalizing 6-DoF Grasp Detection via Domain Prior Knowledgecited by 11
  38. 2022SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulationcited by 10

50 most cited of 138 All 138 (JSON)