mlsanityData to 14 Oct 2026

Hierarchical Deep Deterministic Policy Gradient for Autonomous Maze Navigation of Mobile Robots

arXiv 2508.04994 ↗First posted 7 Aug 2025cs.RO cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Maze navigation is a fundamental challenge in robotics, requiring agents to traverse complex environments efficiently. While the Deep Deterministic Policy Gradient (DDPG) algorithm excels in control tasks, its performance in maze navigation suffers from sparse rewards, inefficient exploration, and long-horizon planning difficulties, often leading to low success rates and average rewards, sometimes even failing to achieve effective navigation. To address these limitations, this paper proposes an efficient Hierarchical DDPG (HDDPG) algorithm, which includes high-level and low-level policies. The high-level policy employs an advanced DDPG framework to generate intermediate subgoals from a long-term perspective and on a higher temporal scale. The low-level policy, also powered by the improved DDPG algorithm, generates primitive actions by observing current states and following the subgoal assigned by the high-level policy. The proposed method enhances stability with off-policy correction, refining subgoal assignments by relabeling historical experiences. Additionally, adaptive parameter space noise is utilized to improve exploration, and a reshaped intrinsic-extrinsic reward function is employed to boost learning efficiency. Further optimizations, including gradient clipping and Xavier initialization, are employed to improve robustness. The proposed algorithm is rigorously evaluated through numerical simulation experiments executed using the Robot Operating System (ROS) and Gazebo. Regarding the three distinct final targets in autonomous maze navigation tasks, HDDPG significantly overcomes the limitations of standard DDPG and its variants, improving the success rate by at least 56.59% and boosting the average reward by a minimum of 519.03 compared to baseline algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 43 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

15 of 43 references.

References · 15

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  3. 2017Solving high-dimensional partial differential equations using deep learningcited by 556
  4. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  5. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 366
  6. 2017Parameter Space Noise for Explorationcited by 161
  7. 2017Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigationcited by 98
  8. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  9. 2020Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errorscited by 85
  10. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  11. 2018Deep Reinforcement Learning for Autonomous Drivingcited by 36
  12. 2018Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learningcited by 14
3 more
  1. 2021Hierarchies of Planning and Reinforcement Learning for Robot Navigationcited by 11
  2. 2022Generalization in Deep Reinforcement Learning for Robotic Navigation by Reward Shapingcited by 10
  3. 2022AACHER: Assorted Actor-Critic Deep Reinforcement Learning with Hindsight Experience Replaycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning
  2. 2026Path Planning Using Deep Deterministic Policy Gradient: A Reinforcement Learning Approach
  3. 2025Hybrid Motion Planning with Deep Reinforcement Learning for Mobile Robot Navigation
  4. 2026A Comparison of Reinforcement Learning and Optimal Control Methods for Path Planning
  5. 2024HDPlanner: Advancing Autonomous Deployments in Unknown Environments through Hierarchical Decision Networks
  6. 2024ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control
  7. 2026Local-Minimum Escaper: Programmatic Subgoal Generation for Robust Navigation in Unknown Environments ↗
  8. 2024Deep Reinforcement Learning with Enhanced PPO for Safe Mobile Robot Navigation