# Pick and Place Without Geometric Object Models

arXiv: https://arxiv.org/abs/1707.05615 · HTML: https://mlsanity.com/paper/1707.05615/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2017.

## Built on by (most cited first, 9 of 9)

- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [Visual Room Rearrangement](https://mlsanity.com/paper/2103.16544/) (2021)
- [Long-Horizon Manipulation of Unknown Objects via Task and Motion Planning with Estimated Affordances](https://mlsanity.com/paper/2108.04145/) (2021)
- [Rapid Motor Adaptation for Robotic Manipulator Arms](https://mlsanity.com/paper/2312.04670/) (2023)
- [ReorientBot: Learning Object Reorientation for Specific-Posed Placement](https://mlsanity.com/paper/2202.11092/) (2022)
- [Multi-Task Learning with Sequence-Conditioned Transporter Networks](https://mlsanity.com/paper/2109.07578/) (2021)
- [Learning to Grasp on the Moon from 3D Octree Observations with Deep Reinforcement Learning](https://mlsanity.com/paper/2208.00818/) (2022)
- [NIFT: Neural Interaction Field and Template for Object Manipulation](https://mlsanity.com/paper/2210.10992/) (2022)
- [Predicting and Attending to Damaging Collisions for Placing Everyday Objects in Photo-Realistic Simulations](https://mlsanity.com/paper/2102.06507/) (2021)

## Abstract (arXiv)

We propose a novel formulation of robotic pick and place as a deep reinforcement learning (RL) problem. Whereas most deep RL approaches to robotic manipulation frame the problem in terms of low level states and actions, we propose a more abstract formulation. In this formulation, actions are target reach poses for the hand and states are a history of such reaches. We show this approach can solve a challenging class of pick-place and regrasping problems where the exact geometry of the objects to be handled is unknown. The only information our method requires is: 1) the sensor perception available to the robot at test time; 2) prior knowledge of the general class of objects for which the system was trained. We evaluate our method using objects belonging to two different categories, mugs and bottles, both in simulation and on real hardware. Results show a major improvement relative to a shape primitives baseline.