# Sim-to-Real Deep Reinforcement Learning with Manipulators for Pick-and-place

arXiv: https://arxiv.org/abs/2309.09247 · HTML: https://mlsanity.com/paper/2309.09247/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 17 Sep 2023.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Learning a visuomotor controller for real world robotic grasping using simulated depth images](https://mlsanity.com/paper/1706.04652/) (2017)
- [Adapting Deep Visuomotor Representations with Weak Pairwise Constraints](https://mlsanity.com/paper/1511.07111/) (2015)
- [DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition](https://mlsanity.com/paper/1702.08558/) (2017)
- [SafeAPT: Safe Simulation-to-Real Robot Learning using Diverse Policies Learned in Simulation](https://mlsanity.com/paper/2201.13248/) (2022)
- [Federated Reinforcement Learning for Collective Navigation of Robotic Swarms](https://mlsanity.com/paper/2202.01141/) (2022)
- [Sample-efficient Deep Reinforcement Learning with Imaginary Rollouts for Human-Robot Interaction](https://mlsanity.com/paper/1908.05546/) (2019)
- [Sim-and-Real Reinforcement Learning for Manipulation: A Consensus-based Approach](https://mlsanity.com/paper/2302.13423/) (2023)

## Abstract (arXiv)

When transferring a Deep Reinforcement Learning model from simulation to the real world, the performance could be unsatisfactory since the simulation cannot imitate the real world well in many circumstances. This results in a long period of fine-tuning in the real world. This paper proposes a self-supervised vision-based DRL method that allows robots to pick and place objects effectively and efficiently when directly transferring a training model from simulation to the real world. A height-sensitive action policy is specially designed for the proposed method to deal with crowded and stacked objects in challenging environments. The training model with the proposed approach can be applied directly to a real suction task without any fine-tuning from the real world while maintaining a high suction success rate. It is also validated that our model can be deployed to suction novel objects in a real experiment with a suction success rate of 90% without any real-world fine-tuning. The experimental video is available at: https://youtu.be/jSTC-EGsoFA.