mlsanityData to 7 Oct 2026

Exploring Hierarchy-Aware Inverse Reinforcement Learning

arXiv 1807.05037 ↗First posted 13 Jul 2018cs.AI cs.HCcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We introduce a new generative model for human planning under the Bayesian Inverse Reinforcement Learning (BIRL) framework which takes into account the fact that humans often plan using hierarchical strategies. We describe the Bayesian Inverse Hierarchical RL (BIHRL) algorithm for inferring the values of hierarchical planners, and use an illustrative toy model to show that BIHRL retains accuracy where standard BIRL fails. Furthermore, BIHRL is able to accurately predict the goals of `Wikispeedia' game players, with inclusion of hierarchical structure in the model resulting in a large boost in accuracy. We show that BIHRL is able to significantly outperform BIRL even when we only have a weak prior on the hierarchical structure of the plans available to the agent, and discuss the significant challenges that remain for scaling up this framework to more realistic settings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 3 of its 22 references that are in the map. First posted on arXiv 13 Jul 2018.

Builds on

3 of 22 references.

References · 3

  1. 2015Learning the Preferences of Ignorant, Inconsistent Agentscited by 45
  2. 2015Modeling Human Understanding of Complex Intentional Action with a Bayesian Nonparametric Subgoal Modelcited by 2
  3. 2016An Alternative Softmax Operator for Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
  2. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning
  3. 2021A Hierarchical Bayesian model for Inverse RL in Partially-Controlled Environments
  4. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  5. 2024Direct Preference Optimization for Primitive-Enabled Hierarchical RL: A Bilevel Approach
  6. 2024Offline Hierarchical Reinforcement Learning via Inverse Optimization
  7. 2023Curricular Subgoals for Inverse Reinforcement Learning
  8. 2026Interactive Inverse Reinforcement Learning of Interaction Scenarios via Bi-level Optimization