mlsanityData to 7 Oct 2026

Human-Interactive Subgoal Supervision for Efficient Inverse Reinforcement Learning

arXiv 1806.08479 ↗First posted 22 Jun 2018cs.HC cs.AIcited by 2

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Humans are able to understand and perform complex tasks by strategically structuring the tasks into incremental steps or subgoals. For a robot attempting to learn to perform a sequential task with critical subgoal states, such states can provide a natural opportunity for interaction with a human expert. This paper analyzes the benefit of incorporating a notion of subgoals into Inverse Reinforcement Learning (IRL) with a Human-In-The-Loop (HITL) framework. The learning process is interactive, with a human expert first providing input in the form of full demonstrations along with some subgoal states. These subgoal states define a set of subtasks for the learning agent to complete in order to achieve the final goal. The learning agent queries for partial demonstrations corresponding to each subtask as needed when the agent struggles with the subtask. The proposed Human Interactive IRL (HI-IRL) framework is evaluated on several discrete path-planning tasks. We demonstrate that subgoal-based interactive structuring of the learning task results in significantly more efficient learning, requiring only a fraction of the demonstration data needed for learning the underlying reward function with the baseline IRL model.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 5 of its 16 references that are in the map. First posted on arXiv 22 Jun 2018.

Builds on

5 of 16 references.

References · 5

  1. 2016Cooperative Inverse Reinforcement Learningcited by 298
  2. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  3. 2017Repeated Inverse Reinforcement Learningcited by 29
  4. 2016HIRL: Hierarchical Inverse Reinforcement Learning for Long-Horizon Tasks with Delayed Rewardscited by 15
  5. 2016Incorporating Human Domain Knowledge into Large Scale Cost Function Learningcited by 5

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 0; 2022: 0; 2023: 0; 2024: 1.

2 papers.

  1. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
  2. 2024SEAL: SEmantic-Augmented Imitation Learning via Language Modelcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2020Interaction-limited Inverse Reinforcement Learning
  2. 2023Curricular Subgoals for Inverse Reinforcement Learning
  3. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations
  4. 2023Inverse Reinforcement Learning without Reinforcement Learning
  5. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
  6. 2022Reward Learning using Structural Motifs in Inverse Reinforcement Learning
  7. 2022Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience
  8. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learning