# CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning

arXiv: https://arxiv.org/abs/2309.17046 · HTML: https://mlsanity.com/paper/2309.17046/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2023.

## Built on by (most cited first, 8 of 8)

- [Learning to Transfer Human Hand Skills for Robot Manipulations](https://mlsanity.com/paper/2501.04169/) (2025)
- [AAMDM: Accelerated Auto-regressive Motion Diffusion Model](https://mlsanity.com/paper/2401.06146/) (2023)
- [Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2404.17687/) (2024)
- [Reinforcement Learning from Wild Animal Videos](https://mlsanity.com/paper/2412.04273/) (2024)
- [MoReFlow: Motion Retargeting Learning through Unsupervised Flow Matching](https://mlsanity.com/paper/2509.25600/) (2025)
- [Self-Consistent Model-based Adaptation for Visual Reinforcement Learning](https://mlsanity.com/paper/2502.09923/) (2025)
- [One-shot Adaptation of Humanoid Whole-body Motion with Walking Priors](https://mlsanity.com/paper/2510.25241/) (2025)
- [Two2Four: Generative Quadruped Puppeteering from Human Motion](https://mlsanity.com/paper/2607.26108/) (2026)

## Abstract (arXiv)

Human motion driven control (HMDC) is an effective approach for generating natural and compelling robot motions while preserving high-level semantics. However, establishing the correspondence between humans and robots with different body structures is not straightforward due to the mismatches in kinematics and dynamics properties, which causes intrinsic ambiguity to the problem. Many previous algorithms approach this motion retargeting problem with unsupervised learning, which requires the prerequisite skill sets. However, it will be extremely costly to learn all the skills without understanding the given human motions, particularly for high-dimensional robots. In this work, we introduce CrossLoco, a guided unsupervised reinforcement learning framework that simultaneously learns robot skills and their correspondence to human motions. Our key innovation is to introduce a cycle-consistency-based reward term designed to maximize the mutual information between human motions and robot states. We demonstrate that the proposed framework can generate compelling robot motions by translating diverse human motions, such as running, hopping, and dancing. We quantitatively compare our CrossLoco against the manually engineered and unsupervised baseline algorithms along with the ablated versions of our framework and demonstrate that our method translates human motions with better accuracy, diversity, and user preference. We also showcase its utility in other applications, such as synthesizing robot movements from language input and enabling interactive robot control.