# LIPM-Guided Reinforcement Learning for Stable and Perceptive Locomotion in Bipedal Robots

arXiv: https://arxiv.org/abs/2509.09106 · HTML: https://mlsanity.com/paper/2509.09106/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [Humanoid Parkour Learning](https://mlsanity.com/paper/2406.10759/) (2024)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion](https://mlsanity.com/paper/2202.05481/) (2022)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [Learning Humanoid Locomotion with Perceptive Internal Model](https://mlsanity.com/paper/2411.14386/) (2024)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [Learning Vision-Based Bipedal Locomotion for Challenging Terrain](https://mlsanity.com/paper/2309.14594/) (2023)
- [Learning Memory-Based Control for Human-Scale Bipedal Locomotion](https://mlsanity.com/paper/2006.02402/) (2020)
- [CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion](https://mlsanity.com/paper/2405.10830/) (2024)
- [PIE: Parkour with Implicit-Explicit Learning Framework for Legged Robots](https://mlsanity.com/paper/2408.13740/) (2024)
- [RobotKeyframing: Learning Locomotion with High-Level Objectives via Mixture of Dense and Sparse Rewards](https://mlsanity.com/paper/2407.11562/) (2024)
- [Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion](https://mlsanity.com/paper/2408.02662/) (2024)
- [Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator](https://mlsanity.com/paper/2412.03012/) (2024)
- [Template Model Inspired Task Space Learning for Robust Bipedal Locomotion](https://mlsanity.com/paper/2309.15442/) (2023)
- [Walking with Terrain Reconstruction: Learning to Traverse Risky Sparse Footholds](https://mlsanity.com/paper/2409.15692/) (2024)
- [DreamWaQ++: Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning](https://mlsanity.com/paper/2409.19709/) (2024)
- [Contingency Model Predictive Control for Bipedal Locomotion on Moving Surfaces with a Linear Inverted Pendulum Model](https://mlsanity.com/paper/2404.12455/) (2024)
- [Bipedal Robust Walking on Uneven Footholds: Piecewise Slope LIPM with Discrete Model Predictive Control](https://mlsanity.com/paper/2504.02255/) (2025)

## Built on by (most cited first, 2 of 2)

- [Architecture Is All You Need: Diversity-Enabled Sweet Spots for Robust Humanoid Locomotion](https://mlsanity.com/paper/2510.14947/) (2025)
- [Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy](https://mlsanity.com/paper/2605.15517/) (2026)

## Abstract (arXiv)

Achieving stable and robust perceptive locomotion for bipedal robots in unstructured outdoor environments remains a critical challenge due to complex terrain geometry and susceptibility to external disturbances. In this work, we propose a novel reward design inspired by the Linear Inverted Pendulum Model (LIPM) to enable perceptive and stable locomotion in the wild. The LIPM provides theoretical guidance for dynamic balance by regulating the center of mass (CoM) height and the torso orientation. These are key factors for terrain-aware locomotion, as they help ensure a stable viewpoint for the robot's camera. Building on this insight, we design a reward function that promotes balance and dynamic stability while encouraging accurate CoM trajectory tracking. To adaptively trade off between velocity tracking and stability, we leverage the Reward Fusion Module (RFM) approach that prioritizes stability when needed. A double-critic architecture is adopted to separately evaluate stability and locomotion objectives, improving training efficiency and robustness. We validate our approach through extensive experiments on a bipedal robot in both simulation and real-world outdoor environments. The results demonstrate superior terrain adaptability, disturbance rejection, and consistent performance across a wide range of speeds and perceptual conditions.