# Learning to Regrasp by Learning to Place

arXiv: https://arxiv.org/abs/2109.08817 · HTML: https://mlsanity.com/paper/2109.08817/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 18 Sep 2021.

## Builds on (most cited first, 13 of 13)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Deep Learning for Detecting Robotic Grasps](https://mlsanity.com/paper/1301.3592/) (2013)
- [Data-Driven Grasp Synthesis - A Survey](https://mlsanity.com/paper/1309.2660/) (2013)
- [Motion-based Object Segmentation based on Dense RGB-D Scene Flow](https://mlsanity.com/paper/1804.05195/) (2018)
- [Object Placement Planning and Optimization for Robot Manipulators](https://mlsanity.com/paper/1907.02555/) (2019)
- [Design and Control of Roller Grasper V2 for In-Hand Manipulation](https://mlsanity.com/paper/2004.08499/) (2020)
- [Analyzing the Utility of a Support Pin in Sequential Robotic Manipulation](https://mlsanity.com/paper/1512.04891/) (2015)
- [Regrasping by Fixtureless Fixturing](https://mlsanity.com/paper/1809.08522/) (2018)
- [Dual-Arm In-Hand Manipulation and Regrasping Using Dexterous Manipulation Graphs](https://mlsanity.com/paper/1904.11382/) (2019)
- [Planning to Repose Long and Heavy Objects Considering a Combination of Regrasp and Constrained Drooping](https://mlsanity.com/paper/2101.10163/) (2021)

## Built on by (most cited first, 11 of 11)

- [SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields](https://mlsanity.com/paper/2211.09786/) (2022)
- [Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangement](https://mlsanity.com/paper/2307.04751/) (2023)
- [Generative Category-Level Shape and Pose Estimation with Semantic Primitives](https://mlsanity.com/paper/2210.01112/) (2022)
- [ReorientDiff: Diffusion Model based Reorientation for Object Manipulation](https://mlsanity.com/paper/2303.12700/) (2023)
- [RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment](https://mlsanity.com/paper/2112.10143/) (2021)
- [ReorientBot: Learning Object Reorientation for Specific-Posed Placement](https://mlsanity.com/paper/2202.11092/) (2022)
- [IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes](https://mlsanity.com/paper/2112.05298/) (2021)
- [Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Prior](https://mlsanity.com/paper/2402.15402/) (2024)
- [Multi-Object Graph Affordance Network: Goal-Oriented Planning through Learned Compound Object Affordances](https://mlsanity.com/paper/2309.10426/) (2023)
- [6-DoF Stability Field via Diffusion Models](https://mlsanity.com/paper/2310.17649/) (2023)
- [Adapting Skills to Novel Grasps: A Self-Supervised Approach](https://mlsanity.com/paper/2408.00178/) (2024)

## Abstract (arXiv)

In this paper, we explore whether a robot can learn to regrasp a diverse set of objects to achieve various desired grasp poses. Regrasping is needed whenever a robot's current grasp pose fails to perform desired manipulation tasks. Endowing robots with such an ability has applications in many domains such as manufacturing or domestic services. Yet, it is a challenging task due to the large diversity of geometry in everyday objects and the high dimensionality of the state and action space. In this paper, we propose a system for robots to take partial point clouds of an object and the supporting environment as inputs and output a sequence of pick-and-place operations to transform an initial object grasp pose to the desired object grasp poses. The key technique includes a neural stable placement predictor and a regrasp graph-based solution through leveraging and changing the surrounding environment. We introduce a new and challenging synthetic dataset for learning and evaluating the proposed approach. We demonstrate the effectiveness of our proposed system with both simulator and real-world experiments. More videos and visualization examples are available on our project webpage.