# Gait-Conditioned Reinforcement Learning with Multi-Phase Curriculum for Humanoid Locomotion

arXiv: https://arxiv.org/abs/2505.20619 · HTML: https://mlsanity.com/paper/2505.20619/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 27 May 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies](https://mlsanity.com/paper/1905.09808/) (2019)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions](https://mlsanity.com/paper/2203.15103/) (2022)
- [Learning Symmetric and Low-energy Locomotion](https://mlsanity.com/paper/1801.08093/) (2018)
- [ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills](https://mlsanity.com/paper/2005.04323/) (2020)
- [Learning Vision-Based Bipedal Locomotion for Challenging Terrain](https://mlsanity.com/paper/2309.14594/) (2023)
- [Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models](https://mlsanity.com/paper/2308.15143/) (2023)
- [Whole-body Humanoid Robot Locomotion with Human Reference](https://mlsanity.com/paper/2402.18294/) (2024)
- [Sim-to-Real Learning of Footstep-Constrained Bipedal Dynamic Walking](https://mlsanity.com/paper/2203.07589/) (2022)
- [Learning Dynamic Bipedal Walking Across Stepping Stones](https://mlsanity.com/paper/2205.01807/) (2022)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Learning to Adapt through Bio-Inspired Gait Strategies for Versatile Quadruped Locomotion](https://mlsanity.com/paper/2412.09440/) (2024)
- [Learning Bipedal Walking on a Quadruped Robot via Adversarial Motion Priors](https://mlsanity.com/paper/2407.02282/) (2024)

## Built on by (most cited first, 6 of 6)

- [CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions](https://mlsanity.com/paper/2510.14959/) (2025)
- [Reduced-Order Model-Guided Reinforcement Learning for Demonstration-Free Humanoid Locomotion](https://mlsanity.com/paper/2509.19023/) (2025)
- [MyoInteract: A Framework for Fast Prototyping of Biomechanical HCI Tasks using Reinforcement Learning](https://mlsanity.com/paper/2602.15245/) (2026)
- [SCDP: Learning Humanoid Locomotion from Partial Observations via Mixed-Observation Distillation](https://mlsanity.com/paper/2603.09574/) (2026)
- [Multi-Gait Learning for Humanoid Robots Using Reinforcement Learning with Selective Adversarial Motion Prior](https://mlsanity.com/paper/2604.19102/) (2026)
- [GaitSpan: Growing Humanoid Locomotion from Walking to Running](https://mlsanity.com/paper/2607.12114/) (2026)

## Abstract (arXiv)

We present a unified gait-conditioned reinforcement learning framework that enables humanoid robots to perform standing, walking, running, and smooth transitions within a single recurrent policy. A compact reward routing mechanism dynamically activates gait-specific objectives based on a one-hot gait ID, mitigating reward interference and supporting stable multi-gait learning. Human-inspired reward terms promote biomechanically natural motions, such as straight-knee stance and coordinated arm-leg swing, without requiring motion capture data. A structured curriculum progressively introduces gait complexity and expands command space over multiple phases. In simulation, the policy successfully achieves robust standing, walking, running, and gait transitions. On the real Unitree G1 humanoid, we validate standing, walking, and walk-to-stand transitions, demonstrating stable and coordinated locomotion. This work provides a scalable, reference-free solution toward versatile and naturalistic humanoid control across diverse modes and environments.