mlsanityData to 14 Oct 2026

Transferring Experience from Simulation to the Real World for Precise Pick-And-Place Tasks in Highly Cluttered Scenes

arXiv 2101.04781 ↗First posted 12 Jan 2021cs.RO cs.AI cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce a novel learning-based approach for grasping known rigid objects in highly cluttered scenes and precisely placing them based on depth images. Our Placement Quality Network (PQ-Net) estimates the object pose and the quality for each automatically generated grasp pose for multiple objects simultaneously at 92 fps in a single forward pass of a neural network. All grasping and placement trials are executed in a physics simulation and the gained experience is transferred to the real world using domain randomization. We demonstrate that our policy successfully transfers to the real world. PQ-Net outperforms other model-free approaches in terms of grasping success rate and automatically scales to new objects of arbitrary symmetry without any human intervention.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 12 Jan 2021.

Builds on

27 of 44 references.

References · 27

  1. 2016Densely Connected Convolutional Networkscited by 6,720
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  3. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2016YOLO9000: Better, Faster, Strongercited by 1,431
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  7. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  8. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  9. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 241
  10. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  11. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  12. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
15 more
  1. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 144
  2. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  3. 2013Deep Learning for Detecting Robotic Graspscited by 116
  4. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
  5. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 87
  6. 2014Real-Time Grasp Detection Using Convolutional Neural Networkscited by 76
  7. 2018Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS Algorithmscited by 44
  8. 2019Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Pickingcited by 25
  9. 2020Single Shot 6D Object Pose Estimationcited by 19
  10. 2016Defining the Pose of any 3D Rigid Object and an Associated Distancecited by 18
  11. 2017Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learningcited by 15
  12. 2018Multi-View Picking: Next-best-view Reaching for Improved Grasping in Cluttercited by 15
  13. 2018Symmetry Aware Evaluation of 3D Object Detection and Pose Estimation in Scenes of Many Parts in Bulkcited by 12
  14. 2018Real-Time 6D Object Pose Estimation on CPUcited by 7
  15. 2019Improving Data Efficiency of Self-supervised Learning for Robotic Graspingcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 0; 2024: 1.

6 papers.

  1. 2022MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesiscited by 13
  2. 2022Cluttered Food Grasping with Adaptive Fingers and Synthetic-Data Trained Object Detectioncited by 8
  3. 2024Sim-to-Real Grasp Detection with Global-to-Local RGB-D Adaptationcited by 4
  4. 2021Investigations on Output Parameterizations of Neural Networks for Single Shot 6D Object Pose Estimationcited by 3
  5. 2021Precise Object Placement with Pose Distance Estimations for Different Objects and Gripperscited by 0
  6. 2022Evaluating Gaussian Grasp Maps for Generative Grasping Modelscited by 0