# Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment

arXiv: https://arxiv.org/abs/2302.10717 · HTML: https://mlsanity.com/paper/2302.10717/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 23 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 4 of 4)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching](https://mlsanity.com/paper/1710.01330/) (2017)
- [Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning](https://mlsanity.com/paper/1709.06670/) (2017)

## Built on by (most cited first, 13 of 13)

- [Tactile-based Object Retrieval From Granular Media](https://mlsanity.com/paper/2402.04536/) (2024)
- [Precision-Focused Reinforcement Learning Model for Robotic Object Pushing](https://mlsanity.com/paper/2411.08622/) (2024)
- [Corner-Grasp: Multi-Action Grasp Detection and Active Gripper Adaptation for Grasping in Cluttered Environments](https://mlsanity.com/paper/2504.01861/) (2025)
- [Planning for Change: Reinforcement Learning Combined with Bounded Extremum Seeking for Robotic Control under Distribution Shift](https://mlsanity.com/paper/2604.01142/) (2026)
- [Learning visual-based deformable object rearrangement with local graph neural networks](https://mlsanity.com/paper/2310.10307/) (2023)
- [URPlanner: A Universal Paradigm For Collision-Free Robotic Motion Planning Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2505.20175/) (2025)
- [Manipulate-to-Navigate: Reinforcement Learning with Visual Affordances and Manipulability Priors](https://mlsanity.com/paper/2508.13151/) (2025)
- [Growing from Exploration: A self-exploring framework for robots based on foundation models](https://mlsanity.com/paper/2401.13462/) (2024)
- [Learning Manipulation Tasks in Dynamic and Shared 3D Spaces](https://mlsanity.com/paper/2404.17673/) (2024)
- [Accelerated Multi-objective Task Learning using Modified Q-learning Algorithm](https://mlsanity.com/paper/2409.01046/) (2024)
- [Toward Deployable Multi-Robot Collaboration via a Symbolically-Guided Decision Transformer](https://mlsanity.com/paper/2508.13877/) (2025)
- [Learning Object-Centric Spatial Reasoning for Sequential Manipulation in Cluttered Environments](https://mlsanity.com/paper/2603.02511/) (2026)
- [Reinforcement Learning for the Full Strawberry Harvesting Process: Obstacle Separation, Detachment, and Placement](https://mlsanity.com/paper/2607.14708/) (2026)

## Abstract (arXiv)

In this paper, a novel robotic grasping system is established to automatically pick up objects in cluttered scenes. A composite robotic hand composed of a suction cup and a gripper is designed for grasping the object stably. The suction cup is used for lifting the object from the clutter first and the gripper for grasping the object accordingly. We utilize the affordance map to provide pixel-wise lifting point candidates for the suction cup. To obtain a good affordance map, the active exploration mechanism is introduced to the system. An effective metric is designed to calculate the reward for the current affordance map, and a deep Q-Network (DQN) is employed to guide the robotic hand to actively explore the environment until the generated affordance map is suitable for grasping. Experimental results have demonstrated that the proposed robotic grasping system is able to greatly increase the success rate of the robotic grasping in cluttered scenes.