# Interactive Hierarchical Guidance using Language

arXiv: https://arxiv.org/abs/2110.04649 · HTML: https://mlsanity.com/paper/2110.04649/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 9 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Modular Multitask Reinforcement Learning with Policy Sketches](https://mlsanity.com/paper/1611.01796/) (2016)
- [Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces](https://mlsanity.com/paper/1709.10163/) (2017)
- [Using Natural Language for Reward Shaping in Reinforcement Learning](https://mlsanity.com/paper/1903.02020/) (2019)
- [Grounded Language Learning in a Simulated 3D World](https://mlsanity.com/paper/1706.06551/) (2017)
- [Learning to Understand Goal Specifications by Modelling Reward](https://mlsanity.com/paper/1806.01946/) (2018)
- [Exploration--Exploitation in MDPs with Options](https://mlsanity.com/paper/1703.08667/) (2017)
- [A Narration-based Reward Shaping Approach using Grounded Natural Language Commands](https://mlsanity.com/paper/1911.00497/) (2019)
- [Grounding Natural Language Commands to StarCraft II Game States for Narration-Guided Reinforcement Learning](https://mlsanity.com/paper/1906.02671/) (2019)

## Built on by (most cited first, 5 of 5)

- [Improving Intrinsic Exploration with Language Abstractions](https://mlsanity.com/paper/2202.08938/) (2022)
- [LLM Augmented Hierarchical Agents](https://mlsanity.com/paper/2311.05596/) (2023)
- [ReProHRL: Towards Multi-Goal Navigation in the Real World using Hierarchical Agents](https://mlsanity.com/paper/2308.08737/) (2023)
- [Towards an Interpretable Hierarchical Agent Framework using Semantic Goals](https://mlsanity.com/paper/2210.08412/) (2022)
- [SEAL: SEmantic-Augmented Imitation Learning via Language Model](https://mlsanity.com/paper/2410.02231/) (2024)

## Abstract (arXiv)

Reinforcement learning has been successful in many tasks ranging from robotic control, games, energy management etc. In complex real world environments with sparse rewards and long task horizons, sample efficiency is still a major challenge. Most complex tasks can be easily decomposed into high-level planning and low level control. Therefore, it is important to enable agents to leverage the hierarchical structure and decompose bigger tasks into multiple smaller sub-tasks. We introduce an approach where we use language to specify sub-tasks and a high-level planner issues language commands to a low level controller. The low-level controller executes the sub-tasks based on the language commands. Our experiments show that this method is able to solve complex long horizon planning tasks with limited human supervision. Using language has added benefit of interpretability and ability for expert humans to take over the high-level planning task and provide language commands if necessary.