# Learning and Executing Re-usable Behaviour Trees from Natural Language Instruction

arXiv: https://arxiv.org/abs/2106.01650 · HTML: https://mlsanity.com/paper/2106.01650/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 38 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Builds on (most cited first, 5 of 5)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Behavior Trees in Robotics and AI: An Introduction](https://mlsanity.com/paper/1709.00084/) (2017)
- [Deep Reinforcement Learning with a Natural Language Action Space](https://mlsanity.com/paper/1511.04636/) (2015)
- [Robot Navigation in Unseen Spaces using an Abstract Map](https://mlsanity.com/paper/2001.11684/) (2020)
- [A Resourceful Reframing of Behavior Trees](https://mlsanity.com/paper/1803.09099/) (2018)

## Built on by (most cited first, 5 of 5)

- [KT-BT: A Framework for Knowledge Transfer Through Behavior Trees in Multi-Robot Systems](https://mlsanity.com/paper/2209.02886/) (2022)
- [A Study on Training and Developing Large Language Models for Behavior Tree Generation](https://mlsanity.com/paper/2401.08089/) (2024)
- [Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledge](https://mlsanity.com/paper/2202.07138/) (2022)
- [Behaviour Trees for Creating Conversational Explanation Experiences](https://mlsanity.com/paper/2211.06402/) (2022)
- [LLM-Assisted Modeling of Semantic Web-Enabled Multi-Agents Systems with AJAN](https://mlsanity.com/paper/2510.06911/) (2025)

## Abstract (arXiv)

Domestic and service robots have the potential to transform industries such as health care and small-scale manufacturing, as well as the homes in which we live. However, due to the overwhelming variety of tasks these robots will be expected to complete, providing generic out-of-the-box solutions that meet the needs of every possible user is clearly intractable. To address this problem, robots must therefore not only be capable of learning how to complete novel tasks at run-time, but the solutions to these tasks must also be informed by the needs of the user. In this paper we demonstrate how behaviour trees, a well established control architecture in the fields of gaming and robotics, can be used in conjunction with natural language instruction to provide a robust and modular control architecture for instructing autonomous agents to learn and perform novel complex tasks. We also show how behaviour trees generated using our approach can be generalised to novel scenarios, and can be re-used in future learning episodes to create increasingly complex behaviours. We validate this work against an existing corpus of natural language instructions, demonstrate the application of our approach on both a simulated robot solving a toy problem, as well as two distinct real-world robot platforms which, respectively, complete a block sorting scenario, and a patrol scenario.