mlsanityData to 14 Oct 2026

Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region

arXiv 2411.01000 ↗First posted 1 Nov 2024cs.RO cs.SY eess.SYcited by 3

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

This paper introduces a new approach to enhance the robustness of humanoid walking under strong perturbations, such as substantial pushes. Effective recovery from external disturbances requires bipedal robots to dynamically adjust their stepping strategies, including footstep positions and timing. Unlike most advanced walking controllers that restrict footstep locations to a predefined convex region, substantially limiting recoverable disturbances, our method leverages reinforcement learning to dynamically adjust the permissible footstep region, expanding it to a larger, effectively non-convex area and allowing cross-over stepping, which is crucial for counteracting large lateral pushes. Additionally, our method adapts footstep timing in real time to further extend the range of recoverable disturbances. Based on these adjustments, feasible footstep positions and DCM trajectory are planned by solving a QP. Finally, we employ a DCM controller and an inverse dynamics whole-body control framework to ensure the robot effectively follows the trajectory.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 29 references that are in the map. First posted on arXiv 1 Nov 2024.

Builds on

16 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey25 Apr
  2. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control30 Jan

Other references · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  3. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 92
  4. 2020ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skillscited by 54
  5. 2020Learning Task Space Actions for Bipedal Locomotioncited by 21
  6. 2023Learning Bipedal Walking for Humanoids with Current Feedbackcited by 18
  7. 2024Tailoring Solution Accuracy for Fast Whole-body Model Predictive Control of Legged Robotscited by 13
  8. 2022Learning Dynamic Bipedal Walking Across Stepping Stonescited by 9
  9. 2023Template Model Inspired Task Space Learning for Robust Bipedal Locomotioncited by 9
  10. 2020Learning Whole-body Motor Skills for Humanoidscited by 6
  11. 2023Walking-by-Logic: Signal Temporal Logic-Guided Model Predictive Control for Bipedal Locomotion Resilient to External Perturbationscited by 6
  12. 2017Walking Control Based on Step Timing Adaptationcited by 4
2 more
  1. 2021On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learningcited by 3
  2. 2023Reachability Aware Capture Regions with Time Adjustment and Cross-Over for Step Recoverycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervisioncited by 2
  2. 2026HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimickingcited by 0
  3. 2026Stubborn: A Streamlined and Unified Reinforcement Learning Framework for Robust Motion Tracking and Fall Recovery for Humanoidscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion
  2. 2026Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds
  3. 2026Model-Informed Safe Reinforcement Learning for Bipedal Locomotion via Step-to-Step Prediction
  4. 2025Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning
  5. 2026FastStair: Learning to Run Up Stairs with Humanoid Robots ↗
  6. 2024RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control
  7. 2026MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds
  8. 2024FootstepNet: an Efficient Actor-Critic Method for Fast On-line Bipedal Footstep Planning and Forecasting