# Learning natural locomotion behaviors for humanoid robots using human knowledge

arXiv: https://arxiv.org/abs/2005.10195 · HTML: https://mlsanity.com/paper/2005.10195/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 20 May 2020.

## Placed in

- Offline RL and imitation learning from data / [Real-world robot offline RL](https://mlsanity.com/offlinerl/real-world-robot-offline-rl/)

## Built on by (most cited first, 11 of 11)

- [Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning](https://mlsanity.com/paper/2408.14472/) (2024)
- [Learning Bipedal Walking On Planned Footsteps For Humanoid Robots](https://mlsanity.com/paper/2207.12644/) (2022)
- [Sim-to-Real Learning for Bipedal Locomotion Under Unsensed Dynamic Loads](https://mlsanity.com/paper/2204.04340/) (2022)
- [Hierarchical generative modelling for autonomous robots](https://mlsanity.com/paper/2308.07775/) (2023)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Identifying Important Sensory Feedback for Learning Locomotion Skills](https://mlsanity.com/paper/2306.17101/) (2023)
- [Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls](https://mlsanity.com/paper/2405.00693/) (2024)
- [On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning](https://mlsanity.com/paper/2104.14534/) (2021)
- [Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning](https://mlsanity.com/paper/2504.13619/) (2025)
- [Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments](https://mlsanity.com/paper/2101.07599/) (2021)
- [Stabilizing Humanoid Robot Trajectory Generation via Physics-Informed Learning and Control-Informed Steering](https://mlsanity.com/paper/2509.24697/) (2025)

## Abstract (arXiv)

This paper presents a new learning framework that leverages the knowledge from imitation learning, deep reinforcement learning, and control theories to achieve human-style locomotion that is natural, dynamic, and robust for humanoids. We proposed novel approaches to introduce human bias, i.e. motion capture data and a special Multi-Expert network structure. We used the Multi-Expert network structure to smoothly blend behavioral features, and used the augmented reward design for the task and imitation rewards. Our reward design is composable, tunable, and explainable by using fundamental concepts from conventional humanoid control. We rigorously validated and benchmarked the learning framework which consistently produced robust locomotion behaviors in various test scenarios. Further, we demonstrated the capability of learning robust and versatile policies in the presence of disturbances, such as terrain irregularities and external pushes.