mlsanityData to 14 Oct 2026

Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains

arXiv 2405.17227 ↗First posted 27 May 2024cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

This paper addresses the challenge of terrain-adaptive dynamic locomotion in humanoid robots, a problem traditionally tackled by optimization-based methods or reinforcement learning (RL). Optimization-based methods, such as model-predictive control, excel in finding optimal reaction forces and achieving agile locomotion, especially in quadruped, but struggle with the nonlinear hybrid dynamics of legged systems and the real-time computation of step location, timing, and reaction forces. Conversely, RL-based methods show promise in navigating dynamic and rough terrains but are limited by their extensive data requirements. We introduce a novel locomotion architecture that integrates a neural network policy, trained through RL in simplified environments, with a state-of-the-art motion controller combining model-predictive control (MPC) and whole-body impulse control (WBIC). The policy efficiently learns high-level locomotion strategies, such as gait selection and step positioning, without the need for full dynamics simulations. This control architecture enables humanoid robots to dynamically navigate discrete terrains, making strategic locomotion decisions (e.g., walking, jumping, and leaping) based on ground height maps. Our results demonstrate that this integrated control architecture achieves dynamic locomotion with significantly fewer training samples than conventional RL-based methods and can be transferred to different humanoid platforms without additional training. The control architecture has been extensively tested in dynamic simulations, accomplishing terrain height-based dynamic locomotion for three different robots.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 37 of its 64 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

37 of 64 references.

References · 37

  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  2. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  3. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  4. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  5. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  6. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  7. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
  8. 2023Extreme Parkour with Legged Robotscited by 164
  9. 2023ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robotscited by 152
  10. 2023Robot Parkour Learningcited by 149
  11. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  12. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 96
25 more
  1. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 92
  2. 2020RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Controlcited by 59
  3. 2019Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Controlcited by 54
  4. 2020ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skillscited by 54
  5. 2021Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contactcited by 52
  6. 2019DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learningcited by 50
  7. 2021Learning to Jump from Pixelscited by 45
  8. 2023Learning Vision-Based Bipedal Locomotion for Challenging Terraincited by 42
  9. 2022OPT-Mimic: Imitation of Optimized Trajectories for Dynamic Quadruped Behaviorscited by 35
  10. 2023Learning and Adapting Agile Locomotion Skills by Transferring Experiencecited by 33
  11. 2021The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviorscited by 32
  12. 2023HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitationcited by 32
  13. 2022Learning Torque Control for Quadrupedal Locomotioncited by 29
  14. 2023RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotioncited by 26
  15. 2021GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Modelcited by 25
  16. 2020Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotioncited by 24
  17. 2023Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfercited by 22
  18. 2021Fast Contact-Implicit Model-Predictive Controlcited by 16
  19. 2023CAJun: Continuous Adaptive Jumping using a Learned Centroidal Controllercited by 15
  20. 2022Learning Bipedal Walking On Planned Footsteps For Humanoid Robotscited by 12
  21. 2018Stair Climbing Stabilization of the HRP-4 Humanoid Robot using Whole-body Admittance Controlcited by 9
  22. 2019Dynamic Locomotion For Passive-Ankle Biped Robots And Humanoids Using Whole-Body Locomotion Controlcited by 8
  23. 2019MPC-based Controller with Terrain Insight for Dynamic Legged Locomotioncited by 8
  24. 2022Optimizing Bipedal Maneuvers of Single Rigid-Body Models for Reinforcement Learningcited by 6
  25. 2022Tello Leg: The Study of Design Principles and Metrics for Dynamic Humanoid Robotscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RL-augmented Adaptive Model Predictive Control for Bipedal Locomotion over Challenging Terrain
  2. 2025Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
  3. 2026Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds
  4. 2024Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning
  5. 2026Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking
  6. 2025Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning
  7. 2024Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion
  8. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control