# End-to-End Learning to Grasp via Sampling from Object Point Clouds

arXiv: https://arxiv.org/abs/2203.05585 · HTML: https://mlsanity.com/paper/2203.05585/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2022.

## Built on by (most cited first, 15 of 15)

- [GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping](https://mlsanity.com/paper/2403.09637/) (2024)
- [PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation](https://mlsanity.com/paper/2309.15596/) (2023)
- [MonoGraspNet: 6-DoF Grasping with a Single RGB Image](https://mlsanity.com/paper/2209.13036/) (2022)
- [ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decomposition](https://mlsanity.com/paper/2403.18062/) (2024)
- [Lightweight Language-driven Grasp Detection using Conditional Consistency Model](https://mlsanity.com/paper/2407.17967/) (2024)
- [PaintNet: Unstructured Multi-Path Learning from 3D Point Clouds for Robotic Spray Painting](https://mlsanity.com/paper/2211.06930/) (2022)
- [Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering](https://mlsanity.com/paper/2306.07392/) (2023)
- [Graph learning in robotics: a survey](https://mlsanity.com/paper/2310.04294/) (2023)
- [GraNet: A Multi-Level Graph Network for 6-DoF Grasp Pose Generation in Cluttered Scenes](https://mlsanity.com/paper/2312.03345/) (2023)
- [Constrained 6-DoF Grasp Generation on Complex Shapes for Improved Dual-Arm Manipulation](https://mlsanity.com/paper/2404.04643/) (2024)
- [Bring Your Own Grasp Generator: Leveraging Robot Grasp Generation for Prosthetic Grasping](https://mlsanity.com/paper/2503.00466/) (2025)
- [MTF-Grasp: A Multi-tier Federated Learning Approach for Robotic Grasping](https://mlsanity.com/paper/2507.10158/) (2025)
- [A comparison of visual representations for real-world reinforcement learning in the context of vacuum gripping](https://mlsanity.com/paper/2503.02405/) (2025)
- [Low Resolution Next Best View for Robot Packing](https://mlsanity.com/paper/2505.04228/) (2025)
- [MultiGraspNet: A Multitask 3D Vision Model for Multi-gripper Robotic Grasping](https://mlsanity.com/paper/2602.06504/) (2026)

## Abstract (arXiv)

The ability to grasp objects is an essential skill that enables many robotic manipulation tasks. Recent works have studied point cloud-based methods for object grasping by starting from simulated datasets and have shown promising performance in real-world scenarios. Nevertheless, many of them still rely on ad-hoc geometric heuristics to generate grasp candidates, which fail to generalize to objects with significantly different shapes with respect to those observed during training. Several approaches exploit complex multi-stage learning strategies and local neighborhood feature extraction while ignoring semantic global information. Furthermore, they are inefficient in terms of number of training samples and time required for inference. In this paper, we propose an end-to-end learning solution to generate 6-DOF parallel-jaw grasps starting from the 3D partial view of the object. Our Learning to Grasp (L2G) method gathers information from the input point cloud through a new procedure that combines a differentiable sampling strategy to identify the visible contact points, with a feature encoder that leverages local and global cues. Overall, L2G is guided by a multi-task objective that generates a diverse set of grasps by optimizing contact point sampling, grasp regression, and grasp classification. With a thorough experimental analysis, we show the effectiveness of L2G as well as its robustness and generalization abilities.