# Learning to Speak and Act in a Fantasy Text Adventure Game

arXiv: https://arxiv.org/abs/1903.03094 · HTML: https://mlsanity.com/paper/1903.03094/

## Key facts

- Cited by 122 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2019.

## Built on by (most cited first, 30 of 122)

- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization](https://mlsanity.com/paper/2212.12017/) (2022)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models](https://mlsanity.com/paper/2106.03521/) (2021)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion](https://mlsanity.com/paper/2203.13224/) (2022)
- [Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning](https://mlsanity.com/paper/1909.01871/) (2019)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [Evaluating Human-Language Model Interaction](https://mlsanity.com/paper/2212.09746/) (2022)
- [MindAgent: Emergent Gaming Interaction](https://mlsanity.com/paper/2309.09971/) (2023)
- [Lifelong Learning of Large Language Model based Agents: A Roadmap](https://mlsanity.com/paper/2501.07278/) (2025)
- [Humanoid Agents: Platform for Simulating Human-like Generative Agents](https://mlsanity.com/paper/2310.05418/) (2023)
- [Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models](https://mlsanity.com/paper/2304.13835/) (2023)
- [Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing](https://mlsanity.com/paper/2104.10097/) (2021)
- [BiasAsker: Measuring the Bias in Conversational AI System](https://mlsanity.com/paper/2305.12434/) (2023)
- [Can Language Models Serve as Text-Based World Simulators?](https://mlsanity.com/paper/2406.06485/) (2024)
- [Improving Intrinsic Exploration with Language Abstractions](https://mlsanity.com/paper/2202.08938/) (2022)
- [Werewolf Arena: A Case Study in LLM Evaluation via Social Deduction](https://mlsanity.com/paper/2407.13943/) (2024)
- [Wordcraft: a Human-AI Collaborative Editor for Story Writing](https://mlsanity.com/paper/2107.07430/) (2021)
- [On Transforming Reinforcement Learning by Transformer: The Development Trajectory](https://mlsanity.com/paper/2212.14164/) (2022)
- [Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence](https://mlsanity.com/paper/2210.07109/) (2022)
- [IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script Generation](https://mlsanity.com/paper/2407.01093/) (2024)
- [Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk](https://mlsanity.com/paper/2401.05033/) (2024)
- [Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbots](https://mlsanity.com/paper/2209.03463/) (2022)
- [Not All Memories are Created Equal: Learning to Forget by Expiring](https://mlsanity.com/paper/2105.06548/) (2021)
- [Reason first, then respond: Modular Generation for Knowledge-infused Dialogue](https://mlsanity.com/paper/2111.05204/) (2021)
- [Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledge](https://mlsanity.com/paper/2112.08619/) (2021)
- [CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models](https://mlsanity.com/paper/2305.11262/) (2023)

## Abstract (arXiv)

We introduce a large scale crowdsourced text adventure game as a research platform for studying grounded dialogue. In it, agents can perceive, emote, and act whilst conducting dialogue with other agents. Models and humans can both act as characters within the game. We describe the results of training state-of-the-art generative and retrieval models in this setting. We show that in addition to using past dialogue, these models are able to effectively use the state of the underlying world to condition their predictions. In particular, we show that grounding on the details of the local environment, including location descriptions, and the objects (and their affordances) and characters (and their previous actions) present within it allows better predictions of agent behavior and dialogue. We analyze the ingredients necessary for successful grounding in this setting, and how each of these factors relate to agents that can talk and act successfully.