# Learning 6-DoF Grasping and Pick-Place Using Attention Focus

arXiv: https://arxiv.org/abs/1806.06134 · HTML: https://mlsanity.com/paper/1806.06134/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2018.

## Built on by (most cited first, 7 of 7)

- [Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation](https://mlsanity.com/paper/2106.12534/) (2021)
- [Rapid Motor Adaptation for Robotic Manipulator Arms](https://mlsanity.com/paper/2312.04670/) (2023)
- [Towards Disturbance-Free Visual Mobile Manipulation](https://mlsanity.com/paper/2112.12612/) (2021)
- [Observe Then Act: Asynchronous Active Vision-Action Model for Robotic Manipulation](https://mlsanity.com/paper/2409.14891/) (2024)
- [Depth-aware Object Segmentation and Grasp Detection for Robotic Picking Tasks](https://mlsanity.com/paper/2111.11114/) (2021)
- [Data-efficient learning of object-centric grasp preferences](https://mlsanity.com/paper/2203.00384/) (2022)
- [Design of an Affordable Prosthetic Arm Equipped with Deep Learning Vision-Based Manipulation](https://mlsanity.com/paper/2103.02099/) (2021)

## Abstract (arXiv)

We address a class of manipulation problems where the robot perceives the scene with a depth sensor and can move its end effector in a space with six degrees of freedom -- 3D position and orientation. Our approach is to formulate the problem as a Markov decision process (MDP) with abstract yet generally applicable state and action representations. Finding a good solution to the MDP requires adding constraints on the allowed actions. We develop a specific set of constraints called hierarchical SE(3) sampling (HSE3S) which causes the robot to learn a sequence of gazes to focus attention on the task-relevant parts of the scene. We demonstrate the effectiveness of our approach on three challenging pick-place tasks (with novel objects in clutter and nontrivial places) both in simulation and on a real robot, even though all training is done in simulation.