# Scaling simulation-to-real transfer by learning composable robot skills

arXiv: https://arxiv.org/abs/1809.10253 · HTML: https://mlsanity.com/paper/1809.10253/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2018.

## Built on by (most cited first, 6 of 6)

- [Influencing Towards Stable Multi-Agent Interactions](https://mlsanity.com/paper/2110.08229/) (2021)
- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control](https://mlsanity.com/paper/2306.09010/) (2023)
- [Deep Reinforcement Learning for Sim-to-Real Policy Transfer of VTOL-UAVs Offshore Docking Operations](https://mlsanity.com/paper/2406.00887/) (2024)
- [A Simple Approach to Continual Learning by Transferring Skill Parameters](https://mlsanity.com/paper/2110.10255/) (2021)
- [Conditionally Combining Robot Skills using Large Language Models](https://mlsanity.com/paper/2310.17019/) (2023)

## Abstract (arXiv)

We present a novel solution to the problem of simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation-reality gap, we learn a set of diverse policies that are parameterized in a way that makes them easily reusable. This diversity and parameterization of low-level skills allows us to find a transferable policy that is able to use combinations and variations of different skills to solve more complex, high-level tasks. In particular, we first use simulation to jointly learn a policy for a set of low-level skills, and a "skill embedding" parameterization which can be used to compose them. Later, we learn high-level policies which actuate the low-level policies via this skill embedding parameterization. The high-level policies encode how and when to reuse the low-level skills together to achieve specific high-level tasks. Importantly, our method learns to control a real robot in joint-space to achieve these high-level tasks with little or no on-robot time, despite the fact that the low-level policies may not be perfectly transferable from simulation to real, and that the low-level skills were not trained on any examples of high-level tasks. We illustrate the principles of our method using informative simulation experiments. We then verify its usefulness for real robotics problems by learning, transferring, and composing free-space and contact motion skills on a Sawyer robot using only joint-space control. We experiment with several techniques for composing pre-learned skills, and find that our method allows us to use both learning-based approaches and efficient search-based planning to achieve high-level tasks using only pre-learned skills.