# Reinforcement learning for non-prehensile manipulation: Transfer from simulation to physical system

arXiv: https://arxiv.org/abs/1803.10371 · HTML: https://mlsanity.com/paper/1803.10371/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2018.

## Built on by (most cited first, 17 of 17)

- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play](https://mlsanity.com/paper/2303.12076/) (2023)
- [GenLoco: Generalized Locomotion Controllers for Quadrupedal Robots](https://mlsanity.com/paper/2209.05309/) (2022)
- [Context is Everything: Implicit Identification for Dynamics Adaptation](https://mlsanity.com/paper/2203.05549/) (2022)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance](https://mlsanity.com/paper/2212.09902/) (2022)
- [Reinforcement Learning with Latent Flow](https://mlsanity.com/paper/2101.01857/) (2021)
- [REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation](https://mlsanity.com/paper/2309.03322/) (2023)
- [Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration](https://mlsanity.com/paper/2308.02459/) (2023)
- [CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objects](https://mlsanity.com/paper/2403.10760/) (2024)
- [RAIDER: Reinforcement-aided Spear Phishing Detector](https://mlsanity.com/paper/2105.07582/) (2021)
- [Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions](https://mlsanity.com/paper/2412.13157/) (2024)
- [Precision-Focused Reinforcement Learning Model for Robotic Object Pushing](https://mlsanity.com/paper/2411.08622/) (2024)
- [Distilled Domain Randomization](https://mlsanity.com/paper/2112.03149/) (2021)
- [Closing the Reality Gap: Zero-Shot Sim-to-Real Deployment for Dexterous Force-Based Grasping and Manipulation](https://mlsanity.com/paper/2601.02778/) (2026)
- [Reinforcement Learning for Robotic Safe Control with Force Sensing](https://mlsanity.com/paper/2512.02022/) (2025)
- [On Globally Optimal Stochastic Policy Gradient Methods for Domain Randomized LQR Synthesis](https://mlsanity.com/paper/2603.14197/) (2026)

## Abstract (arXiv)

Reinforcement learning has emerged as a promising methodology for training robot controllers. However, most results have been limited to simulation due to the need for a large number of samples and the lack of automated-yet-safe data collection methods. Model-based reinforcement learning methods provide an avenue to circumvent these challenges, but the traditional concern has been the mismatch between the simulator and the real world. Here, we show that control policies learned in simulation can successfully transfer to a physical system, composed of three Phantom robots pushing an object to various desired target positions. We use a modified form of the natural policy gradient algorithm for learning, applied to a carefully identified simulation model. The resulting policies, trained entirely in simulation, work well on the physical system without additional training. In addition, we show that training with an ensemble of models makes the learned policies more robust to modeling errors, thus compensating for difficulties in system identification.