# Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region

arXiv: https://arxiv.org/abs/2411.01000 · HTML: https://mlsanity.com/paper/2411.01000/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills](https://mlsanity.com/paper/2005.04323/) (2020)
- [Learning Task Space Actions for Bipedal Locomotion](https://mlsanity.com/paper/2011.04741/) (2020)
- [Learning Bipedal Walking for Humanoids with Current Feedback](https://mlsanity.com/paper/2303.03724/) (2023)
- [Tailoring Solution Accuracy for Fast Whole-body Model Predictive Control of Legged Robots](https://mlsanity.com/paper/2407.10789/) (2024)
- [Learning Dynamic Bipedal Walking Across Stepping Stones](https://mlsanity.com/paper/2205.01807/) (2022)
- [Template Model Inspired Task Space Learning for Robust Bipedal Locomotion](https://mlsanity.com/paper/2309.15442/) (2023)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Learning Whole-body Motor Skills for Humanoids](https://mlsanity.com/paper/2002.02991/) (2020)
- [Walking-by-Logic: Signal Temporal Logic-Guided Model Predictive Control for Bipedal Locomotion Resilient to External Perturbations](https://mlsanity.com/paper/2309.13172/) (2023)
- [Walking Control Based on Step Timing Adaptation](https://mlsanity.com/paper/1704.01271/) (2017)
- [On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning](https://mlsanity.com/paper/2104.14534/) (2021)
- [Reachability Aware Capture Regions with Time Adjustment and Cross-Over for Step Recovery](https://mlsanity.com/paper/2307.11968/) (2023)

## Built on by (most cited first, 3 of 3)

- [Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision](https://mlsanity.com/paper/2601.16109/) (2026)
- [HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimicking](https://mlsanity.com/paper/2603.06775/) (2026)
- [Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoids](https://mlsanity.com/paper/2606.12814/) (2026)

## Abstract (arXiv)

This paper introduces a new approach to enhance the robustness of humanoid walking under strong perturbations, such as substantial pushes. Effective recovery from external disturbances requires bipedal robots to dynamically adjust their stepping strategies, including footstep positions and timing. Unlike most advanced walking controllers that restrict footstep locations to a predefined convex region, substantially limiting recoverable disturbances, our method leverages reinforcement learning to dynamically adjust the permissible footstep region, expanding it to a larger, effectively non-convex area and allowing cross-over stepping, which is crucial for counteracting large lateral pushes. Additionally, our method adapts footstep timing in real time to further extend the range of recoverable disturbances. Based on these adjustments, feasible footstep positions and DCM trajectory are planned by solving a QP. Finally, we employ a DCM controller and an inverse dynamics whole-body control framework to ensure the robot effectively follows the trajectory.