# Vision-based Robotic Grasping From Object Localization, Object Pose Estimation to Grasp Estimation for Parallel Grippers: A Review

arXiv: https://arxiv.org/abs/1905.06658 · HTML: https://mlsanity.com/paper/1905.06658/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2019.

## Built on by (most cited first, 12 of 12)

- [Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview](https://mlsanity.com/paper/2105.14291/) (2021)
- [Object Level Depth Reconstruction for Category Level 6D Object Pose Estimation From Monocular RGB Image](https://mlsanity.com/paper/2204.01586/) (2022)
- [StablePose: Learning 6D Object Poses from Geometrically Stable Patches](https://mlsanity.com/paper/2102.09334/) (2021)
- [RGB-based Category-level Object Pose Estimation via Decoupled Metric Scale Recovery](https://mlsanity.com/paper/2309.10255/) (2023)
- [Revisiting Fully Convolutional Geometric Features for Object 6D Pose Estimation](https://mlsanity.com/paper/2307.15514/) (2023)
- [Geometry-Aware Fruit Grasping Estimation for Robotic Harvesting in Orchards](https://mlsanity.com/paper/2112.04363/) (2021)
- [Category-Agnostic 6D Pose Estimation with Conditional Neural Processes](https://mlsanity.com/paper/2206.07162/) (2022)
- [SO(3)-Pose: SO(3)-Equivariance Learning for 6D Object Pose Estimation](https://mlsanity.com/paper/2208.08338/) (2022)
- [Towards Precise Model-free Robotic Grasping with Sim-to-Real Transfer Learning](https://mlsanity.com/paper/2301.12249/) (2023)
- [Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection](https://mlsanity.com/paper/2101.10226/) (2021)
- [Data-efficient learning of object-centric grasp preferences](https://mlsanity.com/paper/2203.00384/) (2022)
- [DcnnGrasp: Towards Accurate Grasp Pattern Recognition with Adaptive Regularizer Learning](https://mlsanity.com/paper/2205.05218/) (2022)

## Abstract (arXiv)

This paper presents a comprehensive survey on vision-based robotic grasping. We conclude three key tasks during vision-based robotic grasping, which are object localization, object pose estimation and grasp estimation. In detail, the object localization task contains object localization without classification, object detection and object instance segmentation. This task provides the regions of the target object in the input data. The object pose estimation task mainly refers to estimating the 6D object pose and includes correspondence-based methods, template-based methods and voting-based methods, which affords the generation of grasp poses for known objects. The grasp estimation task includes 2D planar grasp methods and 6DoF grasp methods, where the former is constrained to grasp from one direction. These three tasks could accomplish the robotic grasping with different combinations. Lots of object pose estimation methods need not object localization, and they conduct object localization and object pose estimation jointly. Lots of grasp estimation methods need not object localization and object pose estimation, and they conduct grasp estimation in an end-to-end manner. Both traditional methods and latest deep learning-based methods based on the RGB-D image inputs are reviewed elaborately in this survey. Related datasets and comparisons between state-of-the-art methods are summarized as well. In addition, challenges about vision-based robotic grasping and future directions in addressing these challenges are also pointed out.