# Hierarchical Decision Making by Generating and Following Natural Language Instructions

arXiv: https://arxiv.org/abs/1906.00744 · HTML: https://mlsanity.com/paper/1906.00744/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 30 of 30)

- [Skill Induction and Planning with Latent Language](https://mlsanity.com/paper/2110.01517/) (2021)
- [Improving Intrinsic Exploration with Language Abstractions](https://mlsanity.com/paper/2202.08938/) (2022)
- [KITE: Keypoint-Conditioned Policies for Semantic Manipulation](https://mlsanity.com/paper/2306.16605/) (2023)
- [Reason first, then respond: Modular Generation for Knowledge-infused Dialogue](https://mlsanity.com/paper/2111.05204/) (2021)
- [LISA: Learning Interpretable Skill Abstractions from Language](https://mlsanity.com/paper/2203.00054/) (2022)
- [Thought Cloning: Learning to Think while Acting by Imitating Human Thinking](https://mlsanity.com/paper/2306.00323/) (2023)
- [Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021](https://mlsanity.com/paper/2205.02388/) (2022)
- [EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RL](https://mlsanity.com/paper/2206.09674/) (2022)
- [Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks](https://mlsanity.com/paper/2210.15629/) (2022)
- [Boosting Search Engines with Interactive Agents](https://mlsanity.com/paper/2109.00527/) (2021)
- [NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment](https://mlsanity.com/paper/2110.06536/) (2021)
- [IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022](https://mlsanity.com/paper/2205.13771/) (2022)
- [MMKGR: Multi-hop Multi-modal Knowledge Graph Reasoning](https://mlsanity.com/paper/2209.01416/) (2022)
- [Perceiving the World: Question-guided Reinforcement Learning for Text-based Games](https://mlsanity.com/paper/2204.09597/) (2022)
- [LLM Augmented Hierarchical Agents](https://mlsanity.com/paper/2311.05596/) (2023)
- [Multitasking Inhibits Semantic Drift](https://mlsanity.com/paper/2104.07219/) (2021)
- [Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.10865/) (2023)
- [Improving Long-Horizon Imitation Through Instruction Prediction](https://mlsanity.com/paper/2306.12554/) (2023)
- [One-Shot Learning from a Demonstration with Hierarchical Latent Language](https://mlsanity.com/paper/2203.04806/) (2022)
- [SAGE: Bridging Semantic and Actionable Parts for GEneralizable Manipulation of Articulated Objects](https://mlsanity.com/paper/2312.01307/) (2023)
- [Guided Policy Search for Parameterized Skills using Adverbs](https://mlsanity.com/paper/2110.15799/) (2021)
- [Teachable Reinforcement Learning via Advice Distillation](https://mlsanity.com/paper/2203.11197/) (2022)
- [Interactive Text Generation](https://mlsanity.com/paper/2303.00908/) (2023)
- [Instructional Text Across Disciplines: A Survey of Representations, Downstream Tasks, and Open Challenges Toward Capable AI Agents](https://mlsanity.com/paper/2410.18529/) (2024)
- [Language and Experience: A Computational Model of Social Learning in Complex Tasks](https://mlsanity.com/paper/2509.00074/) (2025)
- [Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language](https://mlsanity.com/paper/2109.08214/) (2021)
- [Learning Invariable Semantical Representation from Language for Extensible Policy Generalization](https://mlsanity.com/paper/2202.00466/) (2022)
- [Structured Exploration Through Instruction Enhancement for Object Navigation](https://mlsanity.com/paper/2211.08467/) (2022)
- [elsciRL: Integrating Language Solutions into Reinforcement Learning Problem Settings](https://mlsanity.com/paper/2507.08705/) (2025)
- [HERAKLES: Hierarchical Skill Compilation for Open-ended LLM Agents](https://mlsanity.com/paper/2508.14751/) (2025)

## Abstract (arXiv)

We explore using latent natural language instructions as an expressive and compositional representation of complex actions for hierarchical decision making. Rather than directly selecting micro-actions, our agent first generates a latent plan in natural language, which is then executed by a separate model. We introduce a challenging real-time strategy game environment in which the actions of a large number of units must be coordinated across long time scales. We gather a dataset of 76 thousand pairs of instructions and executions from human play, and train instructor and executor models. Experiments show that models using natural language as a latent variable significantly outperform models that directly imitate human actions. The compositional structure of language proves crucial to its effectiveness for action representation. We also release our code, models and data.