# Model-Informed Safe Reinforcement Learning for Bipedal Locomotion via Step-to-Step Prediction

arXiv: https://arxiv.org/abs/2609.34486 · HTML: https://mlsanity.com/paper/2609.34486/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 16 of 16)

- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [3D Underactuated Bipedal Walking via H-LIP based Gait Synthesis and Stepping Stabilization](https://mlsanity.com/paper/2101.09588/) (2021)
- [Template Model Inspired Task Space Learning for Robust Bipedal Locomotion](https://mlsanity.com/paper/2309.15442/) (2023)
- [Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning](https://mlsanity.com/paper/1910.01748/) (2019)
- [Momentum Control with Hierarchical Inverse Dynamics on a Torque-Controlled Humanoid](https://mlsanity.com/paper/1410.7284/) (2014)
- [Zero Dynamics, Pendulum Models, and Angular Momentum in Feedback Control of Bipedal Locomotion](https://mlsanity.com/paper/2105.08170/) (2021)
- [Terrain-Adaptive, ALIP-Based Bipedal Locomotion Controller via Model Predictive Control and Virtual Constraints](https://mlsanity.com/paper/2109.14862/) (2021)
- [Inverse Dynamics Control of Compliant Hybrid Zero Dynamic Walking](https://mlsanity.com/paper/2010.09047/) (2020)
- [Time-Varying ALIP Model and Robust Foot-Placement Control for Underactuated Bipedal Robot Walking on a Swaying Rigid Surface](https://mlsanity.com/paper/2210.13371/) (2022)
- [Safe Whole-Body Task Space Control for Humanoid Robots](https://mlsanity.com/paper/2311.08409/) (2023)
- [Dynamic and Versatile Humanoid Walking via Embedding 3D Actuated SLIP Model with Hybrid LIP Based Stepping](https://mlsanity.com/paper/2008.02435/) (2020)
- [Data-Driven Latent Space Representation for Robust Bipedal Locomotion Learning](https://mlsanity.com/paper/2309.15740/) (2023)
- [SLIP Walking over Rough Terrain via H-LIP Stepping and Backstepping-Barrier Function Inspired Quadratic Program](https://mlsanity.com/paper/2102.09691/) (2021)
- [Resolved Motion Control for 3D Underactuated Bipedal Walking using Linear Inverted Pendulum Dynamics and Neural Adaptation](https://mlsanity.com/paper/2208.01786/) (2022)

## Abstract (arXiv)

Humanoid robots promise versatile mobility in cluttered, human-centric environments, but real deployment demands principled safety. Classical model-based gait generators yield interpretable motions but often lack the robustness and adaptability of modern reinforcement learning (RL) based approaches. We propose a model-informed reinforcement learning framework anchored to the analytical Angular Momentum Linear Inverted Pendulum (ALIP) template. We provide a step-to-step safety certificate for ALIP stepping via a discrete exponential control barrier function (DECBF) and use it as (i) a training-time shaping signal and (ii) a runtime action filter that minimally adjusts swing-foot placement to satisfy template-level constraints. Full-order safety is evaluated empirically on the Digit humanoid in MuJoCo with a whole-body controller stack. Compared to an unconstrained baseline, our approach reduces safety-violation events in the reported external-disturbance trial, while larger lateral-velocity transients reveal a safety-tracking tradeoff.