mlsanityData to 14 Oct 2026

End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation from RGB

arXiv 2107.05287 ↗First posted 12 Jul 2021cs.CV cs.ROcited by 44

Not on the map. Why?

Abstract (arXiv)

In this work, we introduce a novel, end-to-end trainable CNN-based architecture to deliver high quality results for grasp detection suitable for a parallel-plate gripper, and semantic segmentation. Utilizing this, we propose a novel refinement module that takes advantage of previously calculated grasp detection and semantic segmentation and further increases grasp detection accuracy. Our proposed network delivers state-of-the-art accuracy on two popular grasp dataset, namely Cornell and Jacquard. As additional contribution, we provide a novel dataset extension for the OCID dataset, making it possible to evaluate grasp detection in highly challenging scenes. Using this dataset, we show that semantic segmentation can additionally be used to assign grasp candidates to object classes, which can be used to pick specific objects in the scene.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

26 of 40 references.

References · 26

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2017Mask R-CNNcited by 7,277
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  9. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  10. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  11. 2018Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approachcited by 120
  12. 2013Deep Learning for Detecting Robotic Graspscited by 116
14 more
  1. 2019Seamless Scene Segmentationcited by 112
  2. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
  3. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 87
  4. 2017In-Place Activated BatchNorm for Memory-Optimized Training of DNNscited by 81
  5. 2014Real-Time Grasp Detection Using Convolutional Neural Networkscited by 76
  6. 2019Antipodal Robotic Grasping using Generative Residual Convolutional Neural Networkcited by 59
  7. 2019EasyLabel: A Semi-Automatic Pixel-wise Object Annotation Tool for Creating Robotic RGB-D Datasetscited by 58
  8. 2019The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentationcited by 39
  9. 2018ROI-based Robotic Grasp Detection for Object Overlapping Scenescited by 31
  10. 2018Fully Convolutional Grasp Detection Network with Oriented Anchor Boxcited by 27
  11. 2018Densely Supervised Grasp Detector (DSGD)cited by 10
  12. 2019A Single Multi-Task Deep Neural Network with Post-Processing for Object Detection with Reasoning and Robotic Grasp Detectioncited by 8
  13. 2020Scoring Graspability based on Grasp Regression for Better Grasp Predictioncited by 4
  14. 2020Orientation Attentive Robotic Grasp Synthesis with Augmented Grasp Map Representationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 9; 2024: 17; 2025: 10; 2026: 3.

44 papers.

Sort:

By area:

  1. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  2. 2023Grasp-Anything: Large-scale Grasp Dataset from Foundation Modelscited by 33
  3. 2024Language-driven Grasp Detectioncited by 26
  4. 2023Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Cluttercited by 22
  5. 2024Context-Aware Interaction Network for RGB-T Semantic Segmentationcited by 21
  6. 2022TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentationcited by 17
  7. 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidancecited by 16
  8. 2024Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and Drawer Manipulation in Point Cloudscited by 15
  9. 2024Towards Open-World Grasping with Large Vision-Language Modelscited by 15
  10. 2022When Transformer Meets Robotic Grasping: Exploits Context for Efficient Grasp Detectioncited by 12
  11. 2023RGB-D Grasp Detection via Depth Guided Learning with Cross-modal Attentioncited by 9
  12. 2024Lightweight Language-driven Grasp Detection using Conditional Consistency Modelcited by 9
32 more
  1. 2025GraspClutter6D: A Large-scale Real-world Dataset for Robust Perception and Grasping in Cluttered Scenescited by 8
  2. 2023Sim-Suction: Learning a Suction Grasp Policy for Cluttered Environments Using a Synthetic Benchmarkcited by 7
  3. 2023Variational Probabilistic Fusion Network for RGB-T Semantic Segmentationcited by 7
  4. 2024HiFi-CS: Towards Open Vocabulary Visual Grounding For Robotic Grasping Using Vision-Language Modelscited by 7
  5. 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applicationscited by 5
  6. 2023Learning Spatial-Temporal Regularized Tensor Sparse RPCA for Background Subtractioncited by 4
  7. 2025Evaluating SAM2 for Video Semantic Segmentationcited by 4
  8. 2021Depth-aware Object Segmentation and Grasp Detection for Robotic Picking Taskscited by 3
  9. 2024GraspMamba: A Mamba-based Language-driven Grasp Detection Framework with Hierarchical Feature Learningcited by 3
  10. 2022A Robotic Visual Grasping Design: Rethinking Convolution Neural Network with High-Resolutionscited by 2
  11. 2024AGILE: Approach-based Grasp Inference Learned from Element Decompositioncited by 2
  12. 2024Show and Grasp: Few-shot Semantic Segmentation for Robot Grasping through Zero-shot Foundation Modelscited by 2
  13. 2024Language-driven Grasp Detection with Mask-guided Attentioncited by 2
  14. 2024HMT-Grasp: A Hybrid Mamba-Transformer Approach for Robot Grasping in Cluttered Environmentscited by 2
  15. 2025VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generationcited by 2
  16. 2025The SAM2-to-SAM3 Gap in the Segment Anything Model Family: Why Prompt-Based Expertise Fails in Concept-Driven Image Segmentationcited by 2
  17. 2023FViT-Grasp: Grasping Objects With Using Fast Vision Transformerscited by 1
  18. 2024A Brief Survey on Leveraging Large Scale Vision Models for Enhanced Robot Graspingcited by 1
  19. 2025Visual Affordance Prediction: Survey and Reproducibilitycited by 1
  20. 2025Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulationcited by 1
  21. 2026RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulationcited by 1
  22. 2022A Secure and Efficient Multi-Object Grasping Detection Approach for Robotic Armscited by 0
  23. 2023Modular Anti-noise Deep Learning Network for Robotic Grasp Detection Based on RGB Imagescited by 0
  24. 2023LocaliseBot: Multi-view 3D object localisation with differentiable rendering for robot graspingcited by 0
  25. 20243D Feature Distillation with Object-Centric Priorscited by 0
  26. 2024Open-RGBT: Open-vocabulary RGB-T Zero-shot Semantic Segmentation in Open-world Environmentscited by 0
  27. 2025BIMII-Net: Brain-Inspired Multi-Iterative Interactive Network for RGB-T Road Scene Semantic Segmentationcited by 0
  28. 2025Boosting Cross-spectral Unsupervised Domain Adaptation for Thermal Semantic Segmentationcited by 0
  29. 2025SPGrasp: Spatiotemporal Prompt-driven Grasp Synthesis in Dynamic Scenescited by 0
  30. 2025VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Modelscited by 0
  31. 2026Digital Twin Driven Textile Classification and Foreign Object Recognition in Automated Sorting Systemscited by 0
  32. 2026Removing Infrastructure Barriers in Human-Robot Collaboration Through Wireless Reconfigurable Cellscited by 0