# Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning

arXiv: https://arxiv.org/abs/2002.05954 · HTML: https://mlsanity.com/paper/2002.05954/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 6 of 6)

- [D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions](https://mlsanity.com/paper/2112.03028/) (2021)
- [Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation](https://mlsanity.com/paper/2102.11051/) (2021)
- [SFP: State-free Priors for Exploration in Off-Policy Reinforcement Learning](https://mlsanity.com/paper/2205.13528/) (2022)
- [A Critical Review of Traffic Signal Control and A Novel Unified View of Reinforcement Learning and Model Predictive Control Approaches for Adaptive Traffic Signal Control](https://mlsanity.com/paper/2211.14426/) (2022)
- [Efficient Learning of High Level Plans from Play](https://mlsanity.com/paper/2303.09628/) (2023)
- [Query The Agent: Improving sample efficiency through epistemic uncertainty estimation](https://mlsanity.com/paper/2210.02585/) (2022)

## Abstract (arXiv)

We present HiDe, a novel hierarchical reinforcement learning architecture that successfully solves long horizon control tasks and generalizes to unseen test scenarios. Functional decomposition between planning and low-level control is achieved by explicitly separating the state-action spaces across the hierarchy, which allows the integration of task-relevant knowledge per layer. We propose an RL-based planner to efficiently leverage the information in the planning layer of the hierarchy, while the control layer learns a goal-conditioned control policy. The hierarchy is trained jointly but allows for the modular transfer of policy layers across hierarchies of different agents. We experimentally show that our method generalizes across unseen test environments and can scale to 3x horizon length compared to both learning and non-learning based methods. We evaluate on complex continuous control tasks with sparse rewards, including navigation and robot manipulation.