mlsanityData to 14 Oct 2026

Goal Space Abstraction in Hierarchical Reinforcement Learning via Reachability Analysis

arXiv 2309.07168 ↗First posted 12 Sep 2023cs.LG cs.AI cs.FLcited by 0

Not on the map. Why?

Abstract (arXiv)

Open-ended learning benefits immensely from the use of symbolic methods for goal representation as they offer ways to structure knowledge for efficient and transferable learning. However, the existing Hierarchical Reinforcement Learning (HRL) approaches relying on symbolic reasoning are often limited as they require a manual goal representation. The challenge in autonomously discovering a symbolic goal representation is that it must preserve critical information, such as the environment dynamics. In this work, we propose a developmental mechanism for subgoal discovery via an emergent representation that abstracts (i.e., groups together) sets of environment states that have similar roles in the task. We create a HRL algorithm that gradually learns this representation along with the policies and evaluate it on navigation tasks to show the learned representation is interpretable and results in data efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 10 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

8 of 10 references.

References · 8

  1. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  2. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  3. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  4. 2020Generating Adjacency-Constrained Subgoals in Hierarchical Reinforcement Learningcited by 58
  5. 2018Symbol Emergence in Cognitive Developmental Systems: a Surveycited by 39
  6. 2020Hierarchical Affordance Discovery using Intrinsic Motivationcited by 6
  7. 2018Learning a Set of Interrelated Tasks by Using Sequences of Motor Policies for a Strategic Intrinsically Motivated Learnercited by 4
  8. 2022Robots Learn Increasingly Complex Tasks with Intrinsic Motivation and Automatic Curriculum Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Goal Space Abstraction in Hierarchical Reinforcement Learning via Set-Based Reachability Analysis
  2. 2024Reconciling Spatial and Temporal Abstractions for Goal Representation
  3. 2024Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning
  4. 2025Hierarchical Reinforcement Learning in Multi-Goal Spatial Navigation with Autonomous Mobile Robots
  5. 2024Subgoal Discovery Using a Free Energy Paradigm and State Aggregations
  6. 2025Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning
  7. 2026Abstraction for Offline Goal-Conditioned Reinforcement Learning
  8. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?