# Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots

arXiv: https://arxiv.org/abs/2011.05424 · HTML: https://mlsanity.com/paper/2011.05424/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2020.

## Built on by (most cited first, 7 of 7)

- [Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses](https://mlsanity.com/paper/2306.13104/) (2023)
- [Preferential Multi-Objective Bayesian Optimization](https://mlsanity.com/paper/2406.14699/) (2024)
- [Scalable Interactive Machine Learning for Future Command and Control](https://mlsanity.com/paper/2402.06501/) (2024)
- [NaviGait: Navigating Dynamically Feasible Gait Libraries using Deep Reinforcement Learning](https://mlsanity.com/paper/2510.11542/) (2025)
- [A Safe Preference Learning Approach for Personalization with Applications to Autonomous Vehicles](https://mlsanity.com/paper/2311.02099/) (2023)
- [Re-Envisioning Command and Control](https://mlsanity.com/paper/2402.07946/) (2024)
- [Response Time Improves Choice Prediction and Function Estimation for Gaussian Process Models of Perception and Preferences](https://mlsanity.com/paper/2306.06296/) (2023)

## Abstract (arXiv)

This paper presents a framework that leverages both control theory and machine learning to obtain stable and robust bipedal locomotion without the need for manual parameter tuning. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentally -- a process that often requires extensive tuning due to differences between the models and hardware. In this work, the process of gait realization via hybrid zero dynamics (HZD) based optimization is formally combined with preference-based learning to systematically realize dynamically stable walking. Importantly, this learning approach does not require a carefully constructed reward function, but instead utilizes human pairwise preferences. The power of the proposed approach is demonstrated through two experiments on a planar biped AMBER-3M: the first with rigid point-feet, and the second with induced model uncertainty through the addition of springs where the added compliance was not accounted for in the gait generation or in the controller. In both experiments, the framework achieves stable, robust, efficient, and natural walking in fewer than 50 iterations with no reliance on a simulation environment. These results demonstrate a promising step in the unification of control theory and learning.