# Sim-to-Real Learning of Footstep-Constrained Bipedal Dynamic Walking

arXiv: https://arxiv.org/abs/2203.07589 · HTML: https://mlsanity.com/paper/2203.07589/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2022.

## Built on by (most cited first, 13 of 13)

- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Learning Bipedal Walking On Planned Footsteps For Humanoid Robots](https://mlsanity.com/paper/2207.12644/) (2022)
- [Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion](https://mlsanity.com/paper/2408.02662/) (2024)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Gait-Conditioned Reinforcement Learning with Multi-Phase Curriculum for Humanoid Locomotion](https://mlsanity.com/paper/2505.20619/) (2025)
- [Deep Whole-body Parkour](https://mlsanity.com/paper/2601.07701/) (2026)
- [Adaptive Energy Regularization for Autonomous Gait Transition and Energy-Efficient Quadruped Locomotion](https://mlsanity.com/paper/2403.20001/) (2024)
- [AnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Models](https://mlsanity.com/paper/2409.08904/) (2024)
- [Spectral Normalization for Lipschitz-Constrained Policies on Learning Humanoid Locomotion](https://mlsanity.com/paper/2504.08246/) (2025)
- [Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking](https://mlsanity.com/paper/2606.08253/) (2026)
- [MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds](https://mlsanity.com/paper/2606.10288/) (2026)

## Abstract (arXiv)

Recently, work on reinforcement learning (RL) for bipedal robots has successfully learned controllers for a variety of dynamic gaits with robust sim-to-real demonstrations. In order to maintain balance, the learned controllers have full freedom of where to place the feet, resulting in highly robust gaits. In the real world however, the environment will often impose constraints on the feasible footstep locations, typically identified by perception systems. Unfortunately, most demonstrated RL controllers on bipedal robots do not allow for specifying and responding to such constraints. This missing control interface greatly limits the real-world application of current RL controllers. In this paper, we aim to maintain the robust and dynamic nature of learned gaits while also respecting footstep constraints imposed externally. We develop an RL formulation for training dynamic gait controllers that can respond to specified touchdown locations. We then successfully demonstrate simulation and sim-to-real performance on the bipedal robot Cassie. In addition, we use supervised learning to induce a transition model for accurately predicting the next touchdown locations that the controller can achieve given the robot's proprioceptive observations. This model paves the way for integrating the learned controller into a full-order robot locomotion planner that robustly satisfies both balance and environmental constraints.