# Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies

arXiv: https://arxiv.org/abs/1610.06781 · HTML: https://mlsanity.com/paper/1610.06781/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2016.

## Built on by (most cited first, 5 of 5)

- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [Hierarchically Integrated Models: Learning to Navigate from Heterogeneous Robots](https://mlsanity.com/paper/2106.13280/) (2021)
- [PrivilegedDreamer: Explicit Imagination of Privileged Information for Rapid Adaptation of Learned Policies](https://mlsanity.com/paper/2502.11377/) (2025)
- [Machine Learning Meets Advanced Robotic Manipulation](https://mlsanity.com/paper/2309.12560/) (2023)
- [Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning](https://mlsanity.com/paper/2301.13343/) (2023)

## Abstract (arXiv)

While deep learning has had significant successes in computer vision thanks to the abundance of visual data, collecting sufficiently large real-world datasets for robot learning can be costly. To increase the practicality of these techniques on real robots, we propose a modular deep reinforcement learning method capable of transferring models trained in simulation to a real-world robotic task. We introduce a bottleneck between perception and control, enabling the networks to be trained independently, but then merged and fine-tuned in an end-to-end manner to further improve hand-eye coordination. On a canonical, planar visually-guided robot reaching task a fine-tuned accuracy of 1.6 pixels is achieved, a significant improvement over naive transfer (17.5 pixels), showing the potential for more complicated and broader applications. Our method provides a technique for more efficient learning and transfer of visuo-motor policies for real robotic systems without relying entirely on large real-world robot datasets.