# Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies

arXiv: https://arxiv.org/abs/1709.05746 · HTML: https://mlsanity.com/paper/1709.05746/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2017.

## Built on by (most cited first, 11 of 11)

- [Object Detection Using Sim2Real Domain Randomization for Robotic Applications](https://mlsanity.com/paper/2208.04171/) (2022)
- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [An in-depth experimental study of sensor usage and visual reasoning of robots navigating in real environments](https://mlsanity.com/paper/2111.14666/) (2021)
- [Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction](https://mlsanity.com/paper/2306.03857/) (2023)
- [Learning to navigate efficiently and precisely in real environments](https://mlsanity.com/paper/2401.14349/) (2024)
- [Multi-Object Navigation in real environments using hybrid policies](https://mlsanity.com/paper/2401.13800/) (2024)
- [Image-based Navigation in Real-World Environments via Multiple Mid-level Representations: Fusion Models, Benchmark and Efficient Evaluation](https://mlsanity.com/paper/2202.01069/) (2022)
- [HazardNet: Road Debris Detection by Augmentation of Synthetic Models](https://mlsanity.com/paper/2303.07547/) (2023)
- [GoferBot: A Visual Guided Human-Robot Collaborative Assembly System](https://mlsanity.com/paper/2304.08840/) (2023)
- [On Assessing the Usefulness of Proxy Domains for Developing and Evaluating Embodied Agents](https://mlsanity.com/paper/2109.14516/) (2021)
- [Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning](https://mlsanity.com/paper/2301.13343/) (2023)

## Abstract (arXiv)

Various approaches have been proposed to learn visuo-motor policies for real-world robotic applications. One solution is first learning in simulation then transferring to the real world. In the transfer, most existing approaches need real-world images with labels. However, the labelling process is often expensive or even impractical in many robotic applications. In this paper, we propose an adversarial discriminative sim-to-real transfer approach to reduce the cost of labelling real data. The effectiveness of the approach is demonstrated with modular networks in a table-top object reaching task where a 7 DoF arm is controlled in velocity mode to reach a blue cuboid in clutter through visual observations. The adversarial transfer approach reduced the labelled real data requirement by 50%. Policies can be transferred to real environments with only 93 labelled and 186 unlabelled real images. The transferred visuo-motor policies are robust to novel (not seen in training) objects in clutter and even a moving target, achieving a 97.8% success rate and 1.8 cm control accuracy.