Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach
Not on the map. Why?
Abstract (arXiv)
This paper presents a real-time, object-independent grasp synthesis method which can be used for closed-loop grasping. Our proposed Generative Grasping Convolutional Neural Network (GG-CNN) predicts the quality and pose of grasps at every pixel. This one-to-one mapping from a depth image overcomes limitations of current deep-learning grasping techniques by avoiding discrete sampling of grasp candidates and long computation times. Additionally, our GG-CNN is orders of magnitude smaller while detecting stable grasps with equivalent performance to current state-of-the-art techniques. The light-weight and single-pass generative nature of our GG-CNN allows for closed-loop control at up to 50Hz, enabling accurate grasping in non-static environments where objects move and in the presence of robot control inaccuracies. In our real-world tests, we achieve an 83% grasp success rate on a set of previously unseen objects with adversarial geometry and 88% on a set of household objects that are moved during the grasp attempt. We also achieve 81% accuracy when grasping in dynamic clutter.
Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Apr 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 27; 2022: 17; 2023: 24; 2024: 26; 2025: 18; 2026: 5.
117 papers.
Sort:
By area:
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
- 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
- 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
- 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 107
- 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
- 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 59
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
- 2023NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesiscited by 56
- 2024Graspness Discovery in Clutters for Fast and Accurate Grasp Detectioncited by 47
- 2021End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation from RGBcited by 44
- 2022Equivariant Transporter Networkcited by 44
- 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
38 more
- 2024DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenescited by 41
- 2021RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Imagescited by 38
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
- 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
- 2022Edge Grasp Network: A Graph-Based SE(3)-invariant Approach to Grasp Detectioncited by 34
- 2023Learning Human-to-Robot Handovers from Point Cloudscited by 34
- 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
- 2023Grasp-Anything: Large-scale Grasp Dataset from Foundation Modelscited by 33
- 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3Dcited by 32
- 2021Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overviewcited by 31
- 2021DDGC: Generative Deep Dexterous Grasping in Cluttercited by 28
- 2023Integrating Action Knowledge and LLMs for Task Planning and Situation Handling in Open Worldscited by 26
- 2024Language-driven Grasp Detectioncited by 26
- 2022Learning High-DOF Reaching-and-Grasping via Dynamic Representation of Gripper-Object Interactioncited by 23
- 2023Grounding Classical Task Planners via Vision-Language Modelscited by 23
- 2024An Economic Framework for 6-DoF Grasp Detectioncited by 23
- 2023Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Cluttercited by 22
- 2022Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenescited by 21
- 2021DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstrationcited by 20
- 2023Leveraging Symmetries in Pick and Placecited by 18
- 2023Learning 6-DoF Fine-grained Grasp Detection Based on Part Affordance Groundingcited by 17
- 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidancecited by 16
- 2021Simultaneous Semantic and Collision Learning for 6-DoF Grasp Pose Estimationcited by 14
- 2024DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planningcited by 14
- 2023CenterGrasp: Object-Aware Implicit Representation Learning for Simultaneous Shape Reconstruction and 6-DoF Grasp Estimationcited by 13
- 2023GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion Modelscited by 13
- 2024Towards Open-World Grasping with Large Vision-Language Modelscited by 13
- 2025ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Graspingcited by 13
- 2023On Robot Grasp Learning Using Equivariant Modelscited by 12
- 2021REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Cluttercited by 11
- 2024Efficient Heatmap-Guided 6-Dof Grasp Detection in Cluttered Scenescited by 11
- 2024Generalizing 6-DoF Grasp Detection via Domain Prior Knowledgecited by 11
- 2021Simultaneous Multi-View Object Recognition and Grasping in Open-Ended Domainscited by 10
- 2022Model Predictive Control for Fluid Human-to-Robot Handoverscited by 9
- 2022Coarse-to-fine Q-attention with Tree Expansioncited by 9
- 2023RGB-D Grasp Detection via Depth Guided Learning with Cross-modal Attentioncited by 9
- 2023SCENEREPLICA: Benchmarking Real-World Robot Manipulation by Creating Replicable Scenescited by 9
- 2023Security Considerations in AI-Robotics: A Survey of Current Methods, Challenges, and Opportunitiescited by 9
50 most cited of 117 All 117 (JSON)
- 2026Does Imitation Learning Preserve Temporal Robustness in Dexterous Manipulation? An Expert-Learner Comparison Across Task Execution Speeds1 Sep 2026
- 2026Task-Prototype Guided Flow Matching for Few-Shot Generalization in Vision-Language Robot Manipulation18 Aug 2026
- 2026GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors7 Jun 2026
- 2026A Cross-view Fusion Framework for Robust 6-DoF Grasp Pose Estimation5 Jun 2026
- 2026RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulation16 Mar 2026
- 2025Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulation24 Dec 2025
- 2025VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Models8 Nov 2025
- 2025XGrasp: Gripper-Aware Grasp Detection with Multi-Gripper Data Generation13 Oct 2025
- 2025SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streams12 Oct 2025
- 2025VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generation7 Oct 2025
- 2025ImaginationPolicy: Towards Generalizable, Precise and Reliable End-to-End Policy for Robotic Manipulation25 Sep 2025
- 2025GraspFactory: A Large Object-Centric Grasping Dataset24 Sep 2025
38 more
- 2025SPGrasp: Spatiotemporal Prompt-driven Grasp Synthesis in Dynamic Scenes28 Aug 2025
- 2025Optimizing Grasping in Legged Robots: A Deep Learning Approach to Loco-Manipulation24 Aug 2025
- 2025A Multi-Level Similarity Approach for Single-View Object Grasping: Matching, Planning, and Fine-Tuning16 Jul 2025
- 2025Generative Grasp Detection and Estimation with Concept Learning-based Safety Criteria21 Jun 2025
- 2025Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space23 May 2025
- 2025ORACLE-Grasp: Zero-Shot Affordance-Aligned Robotic Grasping using Large Multimodal Models13 May 2025
- 2025Self-Supervised Learning for Robotic Leaf Manipulation: A Hybrid Geometric-Neural Approach6 May 2025
- 2025ZeroGrasp: Zero-Shot Shape Reconstruction Enabled Robotic Grasping15 Apr 2025
- 2025GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructions20 Mar 2025
- 2025A Benchmarking Study of Vision-based Robotic Grasping Algorithms14 Mar 2025
- 2025Coarse-to-Fine 3D Keyframe Transporter3 Feb 2025
- 2024Stereo Hand-Object Reconstruction for Human-to-Robot Handover10 Dec 2024
- 2024RT-Grasp: Reasoning Tuning Robotic Grasping via Multi-modal Large Language Model7 Nov 2024
- 2024DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenes30 Oct 2024
- 2024Single-Shot Learning of Stable Dynamical Systems for Long-Horizon Manipulation Tasks1 Oct 2024
- 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applications26 Sep 2024
- 2024GraspMamba: A Mamba-based Language-driven Grasp Detection Framework with Hierarchical Feature Learning22 Sep 2024
- 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulation21 Aug 2024
- 2024Target-Oriented Object Grasping via Multimodal Human Guidance20 Aug 2024
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations8 Aug 2024
- 2024Language-driven Grasp Detection with Mask-guided Attention29 Jul 2024
- 2024Lightweight Language-driven Grasp Detection using Conditional Consistency Model25 Jul 2024
- 2024Local Occupancy-Enhanced Object Grasping with Multiple Triplanar Projection22 Jul 2024
- 2024Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidance18 Jul 2024
- 2024An Economic Framework for 6-DoF Grasp Detection11 Jul 2024
- 2024Towards Open-World Grasping with Large Vision-Language Models26 Jun 2024
- 2024DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning25 Jun 2024
- 2024Graspness Discovery in Clutters for Fast and Accurate Grasp Detection17 Jun 2024
- 2024Language-driven Grasp Detection13 Jun 2024
- 2024Towards Real-World Efficiency: Domain Randomization in Reinforcement Learning for Pre-Capture of Free-Floating Moving Targets by Autonomous Robots10 Jun 2024
- 2024Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge2 Apr 2024
- 2024Efficient Heatmap-Guided 6-Dof Grasp Detection in Cluttered Scenes27 Mar 2024
- 2024Pseudo-Labeling and Contextual Curriculum Learning for Online Grasp Learning in Robotic Bin Picking4 Mar 2024
- 2024A Collision-Aware Cable Grasping Method in Cluttered Environment22 Feb 2024
- 2024Jacquard V2: Refining Datasets using the Human In the Loop Data Correction Method8 Feb 2024
- 2024Real-time Holistic Robot Pose Estimation with Unknown States8 Feb 2024
- 2024Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D22 Jan 2024
- 2023GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion Models18 Dec 2023
50 newest of 117 All 117 (JSON)
- 2023Grounding Classical Task Planners via Vision-Language Modelscited by 23
- 2024DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planningcited by 14
- 2024Towards Open-World Grasping with Large Vision-Language Modelscited by 13
- 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applicationscited by 5
- 2025ORACLE-Grasp: Zero-Shot Affordance-Aligned Robotic Grasping using Large Multimodal Modelscited by 0
- 2025VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Modelscited by 0
- 2025VLAD-Grasp: Zero-shot Grasp Detection via Vision-Language Models8 Nov 2025
- 2025ORACLE-Grasp: Zero-Shot Affordance-Aligned Robotic Grasping using Large Multimodal Models13 May 2025
- 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applications26 Sep 2024
- 2024Towards Open-World Grasping with Large Vision-Language Models26 Jun 2024
- 2024DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning25 Jun 2024
- 2023Grounding Classical Task Planners via Vision-Language Models17 Apr 2023
- 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 59
- 2023NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesiscited by 56
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
- 2025Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Spacecited by 3
- 2025Coarse-to-Fine 3D Keyframe Transportercited by 2
- 2026Does Imitation Learning Preserve Temporal Robustness in Dexterous Manipulation? An Expert-Learner Comparison Across Task Execution Speedscited by 0
- 2026Does Imitation Learning Preserve Temporal Robustness in Dexterous Manipulation? An Expert-Learner Comparison Across Task Execution Speeds1 Sep 2026
- 2025Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space23 May 2025
- 2025Coarse-to-Fine 3D Keyframe Transporter3 Feb 2025
- 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations8 Aug 2024
- 2023Waypoint-Based Imitation Learning for Robotic Manipulation26 Jul 2023
- 2023NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis18 Jan 2023
- 2024RT-Grasp: Reasoning Tuning Robotic Grasping via Multi-modal Large Language Modelcited by 7
- 2022Kinova Gemini: Interactive Robot Grasping with Visual Reasoning and Conversational AIcited by 2
- 2025GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructionscited by 2
- 2025VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generationcited by 2
- 2022Enhancing Interpretability and Interactivity in Robot Manipulation: A Neurosymbolic Approachcited by 0
- 2025VCoT-Grasp: Grasp Foundation Models with Visual Chain-of-Thought Reasoning for Language-driven Grasp Generation7 Oct 2025
- 2025GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructions20 Mar 2025
- 2024RT-Grasp: Reasoning Tuning Robotic Grasping via Multi-modal Large Language Model7 Nov 2024
- 2022Enhancing Interpretability and Interactivity in Robot Manipulation: A Neurosymbolic Approach3 Oct 2022
- 2022Kinova Gemini: Interactive Robot Grasping with Visual Reasoning and Conversational AI3 Sep 2022
- 2024Language-driven Grasp Detectioncited by 26
- 2023GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion Modelscited by 13
- 2024Lightweight Language-driven Grasp Detection using Conditional Consistency Modelcited by 9
- 2026Task-Prototype Guided Flow Matching for Few-Shot Generalization in Vision-Language Robot Manipulationcited by 0
- 2026Task-Prototype Guided Flow Matching for Few-Shot Generalization in Vision-Language Robot Manipulation18 Aug 2026
- 2024Lightweight Language-driven Grasp Detection using Conditional Consistency Model25 Jul 2024
- 2024Language-driven Grasp Detection13 Jun 2024
- 2023GraspLDM: Generative 6-DoF Grasp Synthesis using Latent Diffusion Models18 Dec 2023
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
- 2021ContactOpt: Optimizing Contact to Improve Graspscited by 137
- 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
- 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 107
- 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
- 2024Graspness Discovery in Clutters for Fast and Accurate Grasp Detectioncited by 47
- 2021End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation from RGBcited by 44
- 2022Equivariant Transporter Networkcited by 44
- 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
- 2024DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenescited by 41
- 2021RGB Matters: Learning 7-DoF Grasp Poses on Monocular RGBD Imagescited by 38
12 most cited of 95 All 95 (JSON)
- 2026GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors7 Jun 2026
- 2026A Cross-view Fusion Framework for Robust 6-DoF Grasp Pose Estimation5 Jun 2026
- 2026RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulation16 Mar 2026
- 2025Language-Guided Grasp Detection with Coarse-to-Fine Learning for Robotic Manipulation24 Dec 2025
- 2025XGrasp: Gripper-Aware Grasp Detection with Multi-Gripper Data Generation13 Oct 2025
- 2025SpikeGrasp: A Benchmark for 6-DoF Grasp Pose Detection from Stereo Spike Streams12 Oct 2025
- 2025ImaginationPolicy: Towards Generalizable, Precise and Reliable End-to-End Policy for Robotic Manipulation25 Sep 2025
- 2025GraspFactory: A Large Object-Centric Grasping Dataset24 Sep 2025
- 2025SPGrasp: Spatiotemporal Prompt-driven Grasp Synthesis in Dynamic Scenes28 Aug 2025
- 2025Optimizing Grasping in Legged Robots: A Deep Learning Approach to Loco-Manipulation24 Aug 2025
- 2025A Multi-Level Similarity Approach for Single-View Object Grasping: Matching, Planning, and Fine-Tuning16 Jul 2025
- 2025Generative Grasp Detection and Estimation with Concept Learning-based Safety Criteria21 Jun 2025
12 newest of 95 All 95 (JSON)