mlsanityData to 7 Oct 2026

Landmark Guided Active Exploration with State-specific Balance Coefficient

arXiv 2306.17484 ↗First posted 30 Jun 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Goal-conditioned hierarchical reinforcement learning (GCHRL) decomposes long-horizon tasks into sub-tasks through a hierarchical framework and it has demonstrated promising results across a variety of domains. However, the high-level policy's action space is often excessively large, presenting a significant challenge to effective exploration and resulting in potentially inefficient training. In this paper, we design a measure of prospect for sub-goals by planning in the goal space based on the goal-conditioned value function. Building upon the measure of prospect, we propose a landmark-guided exploration strategy by integrating the measures of prospect and novelty which aims to guide the agent to explore efficiently and improve sample efficiency. In order to dynamically consider the impact of prospect and novelty on exploration, we introduce a state-specific balance coefficient to balance the significance of prospect and novelty. The experimental results demonstrate that our proposed exploration strategy significantly outperforms the baseline methods across multiple tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

28 of 34 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  6. 2018Exploration by Random Network Distillationcited by 966
  7. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  8. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  9. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  10. 2016Variational Intrinsic Controlcited by 262
  11. 2019Planning with Goal-Conditioned Policiescited by 144
  12. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
16 more
  1. 2018Count-Based Exploration with the Successor Representationcited by 112
  2. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 107
  3. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  4. 2020World Model as a Graph: Learning Latent Landmarks for Planningcited by 67
  5. 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 55
  6. 2018Learning Actionable Representations with Goal-Conditioned Policiescited by 53
  7. 2017Multi-Level Discovery of Deep Optionscited by 50
  8. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learningcited by 48
  9. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  10. 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
  11. 2018Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Explorationcited by 23
  12. 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
  13. 2020NavRep: Unsupervised Representations for Reinforcement Learning of Robot Navigation in Dynamic Human Environmentscited by 12
  14. 2022State-Conditioned Adversarial Subgoal Generationcited by 8
  15. 2021Connecting Deep-Reinforcement-Learning-based Obstacle Avoidance with Conventional Global Planners using Waypoint Generatorscited by 5
  16. 2022MemoNav: Selecting Informative Memories for Visual Navigationcited by 1

Built on by

Not cited yet.