mlsanityData to 14 Oct 2026

Decoupled Hierarchical Reinforcement Learning with State Abstraction for Discrete Grids

arXiv 2506.02050 ↗First posted 1 Jun 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Effective agent exploration remains a core challenge in reinforcement learning (RL) for complex discrete state-space environments, particularly under partial observability. This paper presents a decoupled hierarchical RL framework integrating state abstraction (DcHRL-SA) to address this issue. The proposed method employs a dual-level architecture, consisting of a high level RL-based actor and a low-level rule-based policy, to promote effective exploration. Additionally, state abstraction method is incorporated to cluster discrete states, effectively lowering state dimensionality. Experiments conducted in two discrete customized grid environments demonstrate that the proposed approach consistently outperforms PPO in terms of exploration efficiency, convergence speed, cumulative reward, and policy stability. These results demonstrate a practical approach for integrating decoupled hierarchical policies and state abstraction in discrete grids with large-scale exploration space. Code will be available at https://github.com/XQY169/DcHRL-SA.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 27 references that are in the map. First posted on arXiv 1 Jun 2025.

Builds on

13 of 27 references.

References · 13

  1. 2020Reinforcement Learning with Augmented Datacited by 468
  2. 2016The Option-Critic Architecturecited by 448
  3. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  4. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  5. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  6. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 171
  7. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  8. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  9. 2018PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learningcited by 90
  10. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 70
  11. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 54
  12. 2018Managing engineering systems with large state and action spaces through deep reinforcement learningcited by 32
1 more
  1. 2023Hierarchical Reinforcement Learning Based on Planning Operatorscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Exploring the limits of Hierarchical World Models in Reinforcement Learning
  2. 2026Abstraction for Offline Goal-Conditioned Reinforcement Learning
  3. 2025Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learning
  4. 2024Hierarchical Decision Making Based on Structural Information Principles
  5. 2026Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimization
  6. 2024Reinforcement Learning with Options and State Representation
  7. 2024Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration ↗
  8. 2025Learning with Expert Abstractions for Efficient Multi-Task Continuous Control