Discovering hierarchies using Imitation Learning from hierarchy aware policies
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
Learning options that allow agents to exhibit temporally higher order behavior has proven to be useful in increasing exploration, reducing sample complexity and for various transfer scenarios. Deep Discovery of Options (DDO) is a generative algorithm that learns a hierarchical policy along with options directly from expert trajectories. We perform a qualitative and quantitative analysis of options inferred from DDO in different domains. To this end, we suggest different value metrics like option termination condition, hinge value function error and KL-Divergence based distance metric to compare different methods. Analyzing the termination condition of the options and number of time steps the options were run revealed that the options were terminating prematurely. We suggest modifications which can be incorporated easily and alleviates the problem of shorter options and a collapse of options to the same mode.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 15 references that are in the map. First posted on arXiv 1 Dec 2018.
Builds on
6 of 15 references.
References · 6
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2016The Option-Critic Architecturecited by 448
- 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
- 2018Hierarchical Imitation and Reinforcement Learningcited by 88
- 2017Multi-Level Discovery of Deep Optionscited by 51
- 2017DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrationscited by 43
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021Online Baum-Welch algorithm for Hierarchical Imitation Learning
- 2024Reinforcement Learning with Options and State Representation
- 2021Adversarial Option-Aware Hierarchical Imitation Learning
- 2025Scalable Option Learning in High-Throughput Environments
- 2020Provable Hierarchical Imitation Learning via EM
- 2023Hierarchical Imitation Learning with Vector Quantized Models
- 2026Q-Shaped Options for Hierarchical Reinforcement Learning
- 2024Temporal Abstraction in Reinforcement Learning with Offline Data