mlsanityData to 7 Oct 2026

Proposing Hierarchical Goal-Conditioned Policy Planning in Multi-Goal Reinforcement Learning

arXiv 2501.01727 ↗First posted 3 Jan 2025cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Humanoid robots must master numerous tasks with sparse rewards, posing a challenge for reinforcement learning (RL). We propose a method combining RL and automated planning to address this. Our approach uses short goal-conditioned policies (GCPs) organized hierarchically, with Monte Carlo Tree Search (MCTS) planning using high-level actions (HLAs). Instead of primitive actions, the planning process generates HLAs. A single plan-tree, maintained during the agent's lifetime, holds knowledge about goal achievement. This hierarchy enhances sample efficiency and speeds up reasoning by reusing HLAs and anticipating future actions. Our Hierarchical Goal-Conditioned Policy Planning (HGCPP) framework uniquely integrates GCPs, MCTS, and hierarchical RL, potentially improving exploration and planning in complex tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 3 Jan 2025.

Builds on

18 of 35 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  5. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  6. 2017Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agentscited by 107
  7. 2017A Deeper Look at Experience Replaycited by 104
  8. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 83
  9. 2020Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictorscited by 60
  10. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learningcited by 48
  11. 2016Learning to Perform Physics Experiments via Deep Reinforcement Learningcited by 47
  12. 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
6 more
  1. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  2. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  3. 2021What Can I Do Here? Learning New Skills by Imagining Visual Affordancescited by 22
  4. 2024Probabilistic Subgoal Representations for Hierarchical Reinforcement learningcited by 8
  5. 2023Goal Space Abstraction in Hierarchical Reinforcement Learning via Set-Based Reachability Analysiscited by 6
  6. 2022Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learningcited by 5

Built on by

Not cited yet.