# Template Model Inspired Task Space Learning for Robust Bipedal Locomotion

arXiv: https://arxiv.org/abs/2309.15442 · HTML: https://mlsanity.com/paper/2309.15442/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2023.

## Built on by (most cited first, 9 of 9)

- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control](https://mlsanity.com/paper/2407.17683/) (2024)
- [Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region](https://mlsanity.com/paper/2411.01000/) (2024)
- [LIPM-Guided Reinforcement Learning for Stable and Perceptive Locomotion in Bipedal Robots](https://mlsanity.com/paper/2509.09106/) (2025)
- [Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision](https://mlsanity.com/paper/2601.16109/) (2026)
- [Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation](https://mlsanity.com/paper/2506.02206/) (2025)
- [Learning Terrain Aware Bipedal Locomotion via Reduced Dimensional Perceptual Representations](https://mlsanity.com/paper/2512.12993/) (2025)
- [HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimicking](https://mlsanity.com/paper/2603.06775/) (2026)
- [Model-Informed Safe Reinforcement Learning for Bipedal Locomotion via Step-to-Step Prediction](https://mlsanity.com/paper/2609.34486/) (2026)

## Abstract (arXiv)

This work presents a hierarchical framework for bipedal locomotion that combines a Reinforcement Learning (RL)-based high-level (HL) planner policy for the online generation of task space commands with a model-based low-level (LL) controller to track the desired task space trajectories. Different from traditional end-to-end learning approaches, our HL policy takes insights from the angular momentum-based linear inverted pendulum (ALIP) to carefully design the observation and action spaces of the Markov Decision Process (MDP). This simple yet effective design creates an insightful mapping between a low-dimensional state that effectively captures the complex dynamics of bipedal locomotion and a set of task space outputs that shape the walking gait of the robot. The HL policy is agnostic to the task space LL controller, which increases the flexibility of the design and generalization of the framework to other bipedal robots. This hierarchical design results in a learning-based framework with improved performance, data efficiency, and robustness compared with the ALIP model-based approach and state-of-the-art learning-based frameworks for bipedal locomotion. The proposed hierarchical controller is tested in three different robots, Rabbit, a five-link underactuated planar biped; Walker2D, a seven-link fully-actuated planar biped; and Digit, a 3D humanoid robot with 20 actuated joints. The trained policy naturally learns human-like locomotion behaviors and is able to effectively track a wide range of walking speeds while preserving the robustness and stability of the walking gait even under adversarial conditions.