# Exploration Based Language Learning for Text-Based Games

arXiv: https://arxiv.org/abs/2001.08868 · HTML: https://mlsanity.com/paper/2001.08868/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2020.

## Built on by (most cited first, 18 of 18)

- [Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models](https://mlsanity.com/paper/2405.15143/) (2024)
- [Augmenting Autotelic Agents with Large Language Models](https://mlsanity.com/paper/2305.12487/) (2023)
- [Multi-Stage Episodic Control for Strategic Exploration in Text Games](https://mlsanity.com/paper/2201.01251/) (2022)
- [A Systematic Survey of Text Worlds as Embodied Natural Language Environments](https://mlsanity.com/paper/2107.04132/) (2021)
- [A Survey of Text Games for Reinforcement Learning informed by Natural Language](https://mlsanity.com/paper/2109.09478/) (2021)
- [PDDLEGO: Iterative Planning in Textual Environments](https://mlsanity.com/paper/2405.19793/) (2024)
- [TextWorldExpress: Simulating Text Games at One Million Steps Per Second](https://mlsanity.com/paper/2208.01174/) (2022)
- [Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations](https://mlsanity.com/paper/2405.13828/) (2024)
- [Knowledge-enhanced Agents for Interactive Text Games](https://mlsanity.com/paper/2305.05091/) (2023)
- [Conceptual Reinforcement Learning for Language-Conditioned Tasks](https://mlsanity.com/paper/2303.05069/) (2023)
- [Generative Personas That Behave and Experience Like Humans](https://mlsanity.com/paper/2209.00459/) (2022)
- [Go-Blend behavior and affect](https://mlsanity.com/paper/2109.13388/) (2021)
- [Situated Language Learning via Interactive Narratives](https://mlsanity.com/paper/2103.09977/) (2021)
- [Affordance Extraction with an External Knowledge Database for Text-Based Simulated Environments](https://mlsanity.com/paper/2207.00265/) (2022)
- [Curiosity-Driven LLM-as-a-judge for Personalized Creative Judgment](https://mlsanity.com/paper/2510.05135/) (2025)
- [Taming the Beast: Learning to Control Neural Conversational Models](https://mlsanity.com/paper/2108.10561/) (2021)
- [Automatic Exploration of Textual Environments with Language-Conditioned Autotelic Agents](https://mlsanity.com/paper/2207.04118/) (2022)
- [A Song of Ice and Fire: Analyzing Textual Autotelic Agents in ScienceWorld](https://mlsanity.com/paper/2302.05244/) (2023)

## Abstract (arXiv)

This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games. Text-based computer games describe their world to the player through natural language and expect the player to interact with the game using text. These games are of interest as they can be seen as a testbed for language understanding, problem-solving, and language generation by artificial agents. Moreover, they provide a learning environment in which these skills can be acquired through interactions with an environment rather than using fixed corpora. One aspect that makes these games particularly challenging for learning agents is the combinatorially large action space. Existing methods for solving text-based games are limited to games that are either very simple or have an action space restricted to a predetermined set of admissible actions. In this work, we propose to use the exploration approach of Go-Explore for solving text-based games. More specifically, in an initial exploration phase, we first extract trajectories with high rewards, after which we train a policy to solve the game by imitating these trajectories. Our experiments show that this approach outperforms existing solutions in solving text-based games, and it is more sample efficient in terms of the number of interactions with the environment. Moreover, we show that the learned policy can generalize better than existing solutions to unseen games without using any restriction on the action space.