# Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning

arXiv: https://arxiv.org/abs/1910.01748 · HTML: https://mlsanity.com/paper/1910.01748/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2019.

## Built on by (most cited first, 8 of 8)

- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Adapting Rapid Motor Adaptation for Bipedal Robots](https://mlsanity.com/paper/2205.15299/) (2022)
- [Learning Linear Policies for Robust Bipedal Locomotion on Terrains with Varying Slopes](https://mlsanity.com/paper/2104.01662/) (2021)
- [On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning](https://mlsanity.com/paper/2104.14534/) (2021)
- [Subequivariant Reinforcement Learning Framework for Coordinated Motion Control](https://mlsanity.com/paper/2403.15100/) (2024)
- [An Energy-Saving Snake Locomotion Gait Policy Obtained Using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.04511/) (2021)
- [Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning](https://mlsanity.com/paper/2407.01050/) (2024)
- [Model-Informed Safe Reinforcement Learning for Bipedal Locomotion via Step-to-Step Prediction](https://mlsanity.com/paper/2609.34486/) (2026)

## Abstract (arXiv)

This paper presents a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from the hybrid nature of the walking dynamics and the well-established hybrid zero dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, short training time, and less dependence on prior knowledge. We demonstrate the effectiveness of the proposed method on Cassie, a challenging 3D bipedal robot. The proposed solution produces stable limit walking cycles that can track various walking speed in different directions. Surprisingly, without specifically trained with disturbances to achieve robustness, it also performs robustly against various adversarial forces applied to the torso towards both the forward and the backward directions.