Hierarchical Reinforcement Learning for Collision-Free Locomotion of an Underactuated Biped
On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation
Abstract (arXiv)
A bipedal robot cannot deviate from its path to avoid an obstacle without disturbing its balance, and this coupling is most severe on underactuated platforms such as the biped considered here, which has four actuated joints per leg and no hip or ankle roll. This paper presents a Hierarchical Reinforcement Learning (HRL) framework in which a High-Level (HL) policy observes the robot pose, 36 raycast proximity measurements, moving-obstacle states, and a receding-horizon local goal, and outputs a body-velocity command (vx, vy, ωyaw) every ten control steps, while a velocity-conditioned Low-Level (LL) policy tracks each command through PD-controlled joint targets. Both policies are trained jointly with Soft Actor-Critic (SAC). Because the converged gait is task-agnostic, it is frozen and driven by classical planners over the same command interface, yielding three controlled baselines: SAC+A*, SAC+RRT*, and SAC+APF. Across 100 evaluation trials per method in randomized PyBullet environments, the proposed method reaches the goal in 98.0% of static and 88.0% of dynamic trials, against at most 78.0% and 68.0% for the planner hybrids, with path lengths within 4% of the A* reference, and ablations confirm that each observation channel and reward term contributes materially to this performance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 21 references that are in the map. First posted on arXiv 5 Oct 2026.
Builds on
14 of 21 references.
Closest ancestors · 1
Same direction, newest first
- 2024Learning Bipedal Walking for Humanoid Robots in Challenging Environments with Obstacle Avoidance25 Sep
Other references · 13
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
- 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
- 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
- 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
- 2017Socially Aware Motion Planning with Deep Reinforcement Learningcited by 136
- 2017Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigationcited by 98
- 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 96
- 2021Goal-Driven Autonomous Exploration Through Deep Reinforcement Learningcited by 35
- 2019Hierarchical Reinforcement Learning for Quadruped Locomotioncited by 24
- 2022Learning Bipedal Walking On Planned Footsteps For Humanoid Robotscited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning a Vision-Based Footstep Planner for Hierarchical Walking Control ↗
- 2024Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey
- 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
- 2025Whole-Body Constrained Learning for Legged Locomotion via Hierarchical Optimization
- 2025Constrained Reinforcement Learning for Unstable Point-Feet Bipedal Locomotion Applied to the Bolt Robot
- 2026Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator ↗
- 2024Agile and versatile bipedal robot tracking control through reinforcement learning
- 2024Hierarchical World Models as Visual Whole-Body Humanoid Controllers