mlsanityData to 14 Oct 2026

Reinforcement Learning with Data Bootstrapping for Dynamic Subgoal Pursuit in Humanoid Robot Navigation

arXiv 2506.02206 ↗First posted 2 Jun 2025cs.ROcited by 1

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Safe and real-time navigation is fundamental for humanoid robot applications. However, existing bipedal robot navigation frameworks often struggle to balance computational efficiency with the precision required for stable locomotion. We propose a novel hierarchical framework that continuously generates dynamic subgoals to guide the robot through cluttered environments. Our method comprises a high-level reinforcement learning (RL) planner for subgoal selection in a robot-centric coordinate system and a low-level Model Predictive Control (MPC) based planner which produces robust walking gaits to reach these subgoals. To expedite and stabilize the training process, we incorporate a data bootstrapping technique that leverages a model-based navigation approach to generate a diverse, informative dataset. We validate our method in simulation using the Agility Robotics Digit humanoid across multiple scenarios with random obstacles. Results show that our framework significantly improves navigation success rates and adaptability compared to both the original model-based method and other learning-based methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 35 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

14 of 35 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Drivingcited by 47
  6. 2022Perceptive Locomotion through Nonlinear Model Predictive Controlcited by 38
  7. 2023DRL-VO: Learning to Navigate Through Crowded Dynamic Scenes Using Velocity Obstaclescited by 37
  8. 2020Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learningcited by 30
  9. 2021Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrianscited by 28
  10. 2023Template Model Inspired Task Space Learning for Robust Bipedal Locomotioncited by 9
  11. 2021Integrated Task and Motion Planning for Safe Legged Navigation in Partially Observable Environmentscited by 8
  12. 2024Real-Time Safe Bipedal Robot Navigation using Linear Discrete Control Barrier Functionscited by 4
2 more
  1. 2022A Sequential MPC Approach to Reactive Planning for Bipedal Robotscited by 3
  2. 2024Unified Path and Gait Planning for Safe Bipedal Robot Navigationcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025An Uncertainty-Weighted Decision Transformer for Navigation in Dense, Complex Driving Scenarioscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains
  2. 2026Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds
  3. 2024Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion
  4. 2026MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds
  5. 2026AGILE: A Comprehensive Workflow for Humanoid Loco-Manipulation Learning
  6. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
  7. 2026GuideWalk: Learning Unified Autonomous Navigation and Locomotion for Humanoid Robots across Versatile Terrains ↗
  8. 2025Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning