# Optimizing Bipedal Maneuvers of Single Rigid-Body Models for Reinforcement Learning

arXiv: https://arxiv.org/abs/2207.04163 · HTML: https://mlsanity.com/paper/2207.04163/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2022.

## Built on by (most cited first, 6 of 6)

- [Robust and Versatile Bipedal Jumping Control through Reinforcement Learning](https://mlsanity.com/paper/2302.09450/) (2023)
- [Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion](https://mlsanity.com/paper/2408.02662/) (2024)
- [CLF-RL: Control Lyapunov Function Guided Reinforcement Learning](https://mlsanity.com/paper/2508.09354/) (2025)
- [Chasing Stability: Humanoid Running via Control Lyapunov Function Guided Reinforcement Learning](https://mlsanity.com/paper/2509.19573/) (2025)
- [Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains](https://mlsanity.com/paper/2405.17227/) (2024)
- [Opt2VLA: Force-Aware Vision-Language-Action for Contact-Rich Humanoid Whole-Body Manipulation](https://mlsanity.com/paper/2609.23968/) (2026)

## Abstract (arXiv)

In this work, we propose a method to generate reduced-order model reference trajectories for general classes of highly dynamic maneuvers for bipedal robots for use in sim-to-real reinforcement learning. Our approach is to utilize a single rigid-body model (SRBM) to optimize libraries of trajectories offline to be used as expert references in the reward function of a learned policy. This method translates the model's dynamically rich rotational and translational behaviour to a full-order robot model and successfully transfers to real hardware. The SRBM's simplicity allows for fast iteration and refinement of behaviors, while the robustness of learning-based controllers allows for highly dynamic motions to be transferred to hardware. % Within this work we introduce a set of transferability constraints that amend the SRBM dynamics to actual bipedal robot hardware, our framework for creating optimal trajectories for dynamic stepping, turning maneuvers and jumps as well as our approach to integrating reference trajectories to a reinforcement learning policy. Within this work we introduce a set of transferability constraints that amend the SRBM dynamics to actual bipedal robot hardware, our framework for creating optimal trajectories for a variety of highly dynamic maneuvers as well as our approach to integrating reference trajectories for a high-speed running reinforcement learning policy. We validate our methods on the bipedal robot Cassie on which we were successfully able to demonstrate highly dynamic grounded running gaits up to 3.0 m/s.