# Reinforcement Learning for Picking Cluttered General Objects with Dense Object Descriptors

arXiv: https://arxiv.org/abs/2304.10108 · HTML: https://mlsanity.com/paper/2304.10108/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation](https://mlsanity.com/paper/1806.08756/) (2018)
- [Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?](https://mlsanity.com/paper/1910.03016/) (2019)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching](https://mlsanity.com/paper/1710.01330/) (2017)
- [S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenes](https://mlsanity.com/paper/1910.14218/) (2019)
- [Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data](https://mlsanity.com/paper/2003.01835/) (2020)
- [Robotic Grasp Detection using Deep Convolutional Neural Networks](https://mlsanity.com/paper/1611.08036/) (2016)
- [PyRep: Bringing V-REP to Deep Robot Learning](https://mlsanity.com/paper/1906.11176/) (2019)
- [SuctionNet-1Billion: A Large-Scale Benchmark for Suction Grasping](https://mlsanity.com/paper/2103.12311/) (2021)
- [Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning](https://mlsanity.com/paper/1709.06670/) (2017)
- [Suction Grasp Region Prediction using Self-supervised Learning for Object Picking in Dense Clutter](https://mlsanity.com/paper/1904.07402/) (2019)
- [X-Ray: Mechanical Search for an Occluded Object by Minimizing Support of Learned Occupancy Distributions](https://mlsanity.com/paper/2004.09039/) (2020)

## Built on by (most cited first, 1 of 1)

- [Multi-Agent Transfer Learning via Temporal Contrastive Learning](https://mlsanity.com/paper/2406.01377/) (2024)

## Abstract (arXiv)

Picking cluttered general objects is a challenging task due to the complex geometries and various stacking configurations. Many prior works utilize pose estimation for picking, but pose estimation is difficult on cluttered objects. In this paper, we propose Cluttered Objects Descriptors (CODs), a dense cluttered objects descriptor that can represent rich object structures, and use the pre-trained CODs network along with its intermediate outputs to train a picking policy. Additionally, we train the policy with reinforcement learning, which enable the policy to learn picking without supervision. We conduct experiments to demonstrate that our CODs is able to consistently represent seen and unseen cluttered objects, which allowed for the picking policy to robustly pick cluttered general objects. The resulting policy can pick 96.69% of unseen objects in our experimental environment which is twice as cluttered as the training scenarios.