mlsanityData to 14 Oct 2026

Depth-aware Object Segmentation and Grasp Detection for Robotic Picking Tasks

arXiv 2111.11114 ↗First posted 22 Nov 2021cs.CV cs.ROcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a novel deep neural network architecture for joint class-agnostic object segmentation and grasp detection for robotic picking tasks using a parallel-plate gripper. We introduce depth-aware Coordinate Convolution (CoordConv), a method to increase accuracy for point proposal based object instance segmentation in complex scenes without adding any additional network parameters or computation complexity. Depth-aware CoordConv uses depth data to extract prior information about the location of an object to achieve highly accurate object instance segmentation. These resulting segmentation masks, combined with predicted grasp candidates, lead to a complete scene description for grasping using a parallel-plate gripper. We evaluate the accuracy of grasp detection and instance segmentation on challenging robotic picking datasets, namely Sil'eane and OCIDgrasp, and show the benefit of joint grasp detection and segmentation on a real-world robotic picking task.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 22 Nov 2021.

Builds on

32 of 44 references.

References · 32

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2017Mask R-CNNcited by 7,277
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  6. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  9. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  10. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  11. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 242
  12. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
20 more
  1. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  2. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 127
  3. 2013Deep Learning for Detecting Robotic Graspscited by 116
  4. 2019Seamless Scene Segmentationcited by 112
  5. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
  6. 2020Unseen Object Instance Segmentation for Robotic Environmentscited by 85
  7. 2014Real-Time Grasp Detection Using Convolutional Neural Networkscited by 76
  8. 2018Depth-aware CNN for RGB-D Segmentationcited by 74
  9. 20196-DOF Grasping for Target-driven Object Manipulation in Cluttercited by 70
  10. 2019AdaptIS: Adaptive Instance Selection Networkcited by 64
  11. 2019EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasetscited by 58
  12. 2016Robotic Grasp Detection using Deep Convolutional Neural Networkscited by 47
  13. 2021End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation from RGBcited by 44
  14. 2018Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methodscited by 33
  15. 2018Fully Convolutional Grasp Detection Network with Oriented Anchor Boxcited by 27
  16. 2018Symmetry Aware Evaluation of 3D Object Detection and Pose Estimation in Scenes of Many Parts in Bulkcited by 12
  17. 2018Densely Supervised Grasp Detector (DSGD)cited by 10
  18. 2019A Single Multi-Task Deep Neural Network with Post-Processing for Object Detection with Reasoning and Robotic Grasp Detectioncited by 8
  19. 2018Recurrent Convolutional Fusion for RGB-D Object Recognitioncited by 7
  20. 2018Learning 6-DoF Grasping and Pick-Place Using Attention Focuscited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 2.

3 papers.

  1. 2025H3DP: Triply-Hierarchical Diffusion Policy for Visuomotor Learningcited by 16
  2. 2023Sim-Suction: Learning a Suction Grasp Policy for Cluttered Environments Using a Synthetic Benchmarkcited by 7
  3. 20253D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understandingcited by 2