mlsanityData to 7 Oct 2026

Simultaneous Semantic and Collision Learning for 6-DoF Grasp Pose Estimation

arXiv 2108.02425 ↗First posted 5 Aug 2021cs.RO cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Grasping in cluttered scenes has always been a great challenge for robots, due to the requirement of the ability to well understand the scene and object information. Previous works usually assume that the geometry information of the objects is available, or utilize a step-wise, multi-stage strategy to predict the feasible 6-DoF grasp poses. In this work, we propose to formalize the 6-DoF grasp pose estimation as a simultaneous multi-task learning problem. In a unified framework, we jointly predict the feasible 6-DoF grasp poses, instance semantic segmentation, and collision information. The whole framework is jointly optimized and end-to-end differentiable. Our model is evaluated on large-scale benchmarks as well as the real robot system. On the public dataset, our method outperforms prior state-of-the-art methods by a large margin (+4.08 AP). We also demonstrate the implementation of our model on a real robotic platform and show that the robot can accurately grasp target objects in cluttered scenarios with a high success rate. Project link: https://openbyterobotics.github.io/sscl

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 31 of its 40 references that are in the map. First posted on arXiv 5 Aug 2021.

Builds on

31 of 40 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  5. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  6. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  7. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 352
  8. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  9. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  10. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 259
  11. 2016Volumetric and Multi-View CNNs for Object Classification on 3D Datacited by 240
  12. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 237
19 more
  1. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  2. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  3. 2017Semantic Instance Segmentation with a Discriminative Loss Functioncited by 118
  4. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 117
  5. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 115
  6. 2013Deep Learning for Detecting Robotic Graspscited by 114
  7. 2017Grasp Pose Detection in Point Cloudscited by 107
  8. 2013Data-Driven Grasp Synthesis - A Surveycited by 104
  9. 2019Self-supervised 6D Object Pose Estimation for Robot Manipulationcited by 100
  10. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 86
  11. 2018PointNetGPD: Detecting Grasp Configurations from Point Setscited by 80
  12. 2014Real-Time Grasp Detection Using Convolutional Neural Networkscited by 75
  13. 20196-DOF Grasping for Target-driven Object Manipulation in Cluttercited by 70
  14. 2019S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenescited by 51
  15. 2016Robotic Grasp Detection using Deep Convolutional Neural Networkscited by 47
  16. 2020PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Cloudscited by 39
  17. 2016High precision grasp pose detection in dense cluttercited by 33
  18. 2019A Billion Ways to Grasp: An Evaluation of Grasp Sampling Schemes on a Dense, Physics-based Grasp Data Setcited by 30
  19. 2021GPR: Grasp Pose Refinement Network for Cluttered Scenescited by 13

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 5; 2024: 5.

14 papers.

Sort:

  1. 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 124
  2. 2022LargeKernel3D: Scaling up Kernels in 3D Sparse CNNscited by 71
  3. 2023IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformationcited by 40
  4. 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
  5. 2024An Economic Framework for 6-DoF Grasp Detectioncited by 23
  6. 2022Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenescited by 21
  7. 2023Deep Fusion Transformer Network with Weighted Vector-Wise Keypoints Voting for Robust 6D Object Pose Estimationcited by 16
  8. 2024ICGNet: A Unified Approach for Instance-Centric Graspingcited by 16
  9. 2022Where To Start? Transferring Simple Skills to Complex Environmentscited by 9
  10. 2022Cluttered Food Grasping with Adaptive Fingers and Synthetic-Data Trained Object Detectioncited by 8
  11. 2023GraNet: A Multi-Level Graph Network for 6-DoF Grasp Pose Generation in Cluttered Scenescited by 4
  12. 2024You Only Scan Once: A Dynamic Scene Reconstruction Pipeline for 6-DoF Robotic Grasping of Novel Objectscited by 2
2 more
  1. 2023SKU-Patch: Towards Efficient Instance Segmentation for Unseen Objects in Auto-Storecited by 0
  2. 2024Self-supervised 6-DoF Robot Grasping by Demonstration via Augmented Reality Teleoperation Systemcited by 0