mlsanityData to 14 Oct 2026

Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning

arXiv 2504.13619 ↗First posted 18 Apr 2025cs.ROcited by 3

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

For the deployment of legged robots in real-world environments, it is essential to develop robust locomotion control methods for challenging terrains that may exhibit unexpected deformability and irregularity. In this paper, we explore the application of sim-to-real deep reinforcement learning (RL) for the design of bipedal locomotion controllers for humanoid robots on compliant and uneven terrains. Our key contribution is to show that a simple training curriculum for exposing the RL agent to randomized terrains in simulation can achieve robust walking on a real humanoid robot using only proprioceptive feedback. We train an end-to-end bipedal locomotion policy using the proposed approach, and show extensive real-robot demonstration on the HRP-5P humanoid over several difficult terrains inside and outside the lab environment. Further, we argue that the robustness of a bipedal walking policy can be improved if the robot is allowed to exhibit aperiodic motion with variable stepping frequency. We propose a new control policy to enable modification of the observed clock signal, leading to adaptive gait frequencies depending on the terrain and command velocity. Through simulation experiments, we show the effectiveness of this policy specifically for walking over challenging terrains by controlling swing and stance durations. The code for training and evaluation is available online at https://github.com/rohanpsingh/LearningHumanoidWalking. Demo video is available at https://www.youtube.com/watch?v=ZgfNzGAkk2Q.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 31 references that are in the map. First posted on arXiv 18 Apr 2025.

Builds on

15 of 31 references.

References · 15

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  3. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  4. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 96
  5. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 92
  6. 2020Learning Memory-Based Control for Human-Scale Bipedal Locomotioncited by 28
  7. 2023Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfercited by 22
  8. 2021DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learningcited by 21
  9. 2023Learning Bipedal Walking for Humanoids with Current Feedbackcited by 18
  10. 2022Learning Bipedal Walking On Planned Footsteps For Humanoid Robotscited by 12
  11. 2020Learning natural locomotion behaviors for humanoid robots using human knowledgecited by 11
  12. 2018Stair Climbing Stabilization of the HRP-4 Humanoid Robot using Whole-body Admittance Controlcited by 9
3 more
  1. 2022mc-mujoco: Simulating Articulated Robots with FSM Controllers in MuJoCocited by 5
  2. 2020Inverse Dynamics Control of Compliant Hybrid Zero Dynamic Walkingcited by 3
  3. 2021Learning Linear Policies for Robust Bipedal Locomotion on Terrains with Varying Slopescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Mechanical Intelligence-Aware Curriculum Reinforcement Learning for Humanoids with Parallel Actuationcited by 1
  2. 2025Architecture Is All You Need: Diversity-Enabled Sweet Spots for Robust Humanoid Locomotioncited by 1
  3. 2026Smoothness as a Constraint for Stable Humanoid Locomotioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains
  2. 2024Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning
  3. 2025RL-augmented Adaptive Model Predictive Control for Bipedal Locomotion over Challenging Terrain
  4. 2024Learning Humanoid Locomotion over Challenging Terrain
  5. 2026Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds
  6. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
  7. 2026MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds
  8. 2025Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction