Learning to Speak and Act in a Fantasy Text Adventure Game
Not on the map. Why?
Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.
Abstract (arXiv)
We introduce a large scale crowdsourced text adventure game as a research platform for studying grounded dialogue. In it, agents can perceive, emote, and act whilst conducting dialogue with other agents. Models and humans can both act as characters within the game. We describe the results of training state-of-the-art generative and retrieval models in this setting. We show that in addition to using past dialogue, these models are able to effectively use the state of the underlying world to condition their predictions. In particular, we show that grounding on the details of the local environment, including location descriptions, and the objects (and their affordances) and characters (and their previous actions) present within it allows better predictions of agent behavior and dialogue. We analyze the ingredients necessary for successful grounding in this setting, and how each of these factors relate to agents that can talk and act successfully.
Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Mar 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2019: 1; 2020: 0; 2021: 20; 2022: 27; 2023: 27; 2024: 27; 2025: 13; 2026: 7.
122 papers.
Sort:
By area:
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
- 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
- 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
- 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
- 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
- 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
- 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
- 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
- 2024Large Language Models and Games: A Survey and Roadmapcited by 91
- 2022Evaluating Human-Language Model Interactioncited by 90
- 2023MindAgent: Emergent Gaming Interactioncited by 87
38 more
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
- 2023Humanoid Agents: Platform for Simulating Human-like Generative Agentscited by 73
- 2023Multi-Party Chat: Conversational Agents in Group Settings with Humans and Modelscited by 53
- 2021Beyond Fair Pay: Ethical Implications of NLP Crowdsourcingcited by 52
- 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 52
- 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
- 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
- 2024Werewolf Arena: A Case Study in LLM Evaluation via Social Deductioncited by 42
- 2021Wordcraft: a Human-AI Collaborative Editor for Story Writingcited by 41
- 2022Dungeons and Dragons as a Dialog Challenge for Artificial Intelligencecited by 35
- 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
- 2024IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script Generationcited by 35
- 2024Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talkcited by 34
- 2021Not All Memories are Created Equal: Learning to Forget by Expiringcited by 32
- 2021Reason first, then respond: Modular Generation for Knowledge-infused Dialoguecited by 32
- 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
- 2021Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledgecited by 30
- 2023CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Modelscited by 25
- 2023MPCHAT: Towards Multimodal Persona-Grounded Conversationcited by 24
- 2023LARP: Language-Agent Role Play for Open-World Gamescited by 24
- 2024Security of AI Agentscited by 24
- 2022Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021cited by 23
- 2021Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identitycited by 22
- 2022I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragonscited by 22
- 2023ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Gamescited by 22
- 2024TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Modelscited by 21
- 2023HandMeThat: Human-Robot Communication in Physical and Social Environmentscited by 19
- 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
- 2024Word2World: Generating Stories and Worlds through Large Language Modelscited by 18
- 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
- 2022TVShowGuess: Character Comprehension in Stories as Speaker Guessingcited by 17
- 2024Player-Driven Emergence in LLM-Driven Game Narrativecited by 16
- 2021NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environmentcited by 15
- 2021Towards Socially Intelligent Agents with Mental State Transition and Human Utilitycited by 14
- 2021A Survey of Text Games for Reinforcement Learning informed by Natural Languagecited by 14
- 2022MBTI Personality Prediction for Fictional Characters Using Movie Scriptscited by 14
- 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
- 2021Telling Stories through Multi-User Dialogue by Modeling Character Relationscited by 13
50 most cited of 122 All 122 (JSON)
- 2026World Agent: Can Language Models Keep a World Running?26 Sep 2026
- 2026TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation31 Jul 2026
- 2026Semantic Flow Regularization: Teaching LLMs to Generate Diverse Yet Coherent Responses27 May 2026
- 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse11 May 2026
- 2026NARRA-Gym for Evaluating Interactive Narrative Agents8 May 2026
- 2026Data Selection for Multi-turn Dialogue Instruction Tuning9 Apr 2026
- 2026Beyond Fixed Psychological Personas: State Beats Trait, but Language Models are State-Blind21 Jan 2026
- 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia3 Dec 2025
- 2025GenQuest: An LLM-based Text Adventure Game for Language Learners6 Oct 2025
- 2025PILOT: Steering Synthetic Data Generation with Psychological & Linguistic Output Targeting18 Sep 2025
- 2025Can LLMs Generate High-Quality Task-Specific Conversations?4 Aug 2025
- 2025MDC-R: The Minecraft Dialogue Corpus with Reference27 Jun 2025
38 more
- 2025ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratch4 Jun 2025
- 2025STORY2GAME: Generating (Almost) Everything in an Interactive Fiction Game6 May 2025
- 2025SAGE: A Generic Framework for LLM Safety Evaluation28 Apr 2025
- 2025debug-gym: A Text-Based Environment for Interactive Debugging27 Mar 2025
- 2025Word2Minecraft: Generating 3D Game Levels through Large Language Models18 Mar 2025
- 2025OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs14 Mar 2025
- 2025Digital Player: Evaluating Large Language Models based Human-like Agent in Games28 Feb 2025
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmap13 Jan 2025
- 2024EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents18 Dec 2024
- 2024Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey14 Nov 2024
- 2024MCPDial: A Minecraft Persona-driven Dialogue Dataset29 Oct 2024
- 2024The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends21 Sep 2024
- 2024Drama Engine: A Framework for Narrative Agents21 Aug 2024
- 2024BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model20 Aug 2024
- 2024What if Red Can Talk? Dynamic Dialogue Generation Using Large Language Models29 Jul 2024
- 2024Werewolf Arena: A Case Study in LLM Evaluation via Social Deduction18 Jul 2024
- 2024A LLM Benchmark based on the Minecraft Builder Dialog Agent Task17 Jul 2024
- 2024A Text-to-Game Engine for UGC-Based Role-Playing Games11 Jul 2024
- 2024MINDECHO: Role-Playing Language Agents for Key Opinion Leaders7 Jul 2024
- 2024IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script Generation1 Jul 2024
- 2024Security of AI Agents12 Jun 2024
- 2024Can Language Models Serve as Text-Based World Simulators?10 Jun 2024
- 2024TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models28 May 2024
- 2024Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre11 May 2024
- 2024A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worlds9 May 2024
- 2024Word2World: Generating Stories and Worlds through Large Language Models6 May 2024
- 2024Player-Driven Emergence in LLM-Driven Game Narrative25 Apr 2024
- 2024Missed Connections: Lateral Thinking Puzzles for Large Language Models17 Apr 2024
- 2024Graph Integrated Language Transformers for Next Action Prediction in Complex Phone Calls11 Apr 2024
- 2024"In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning5 Mar 2024
- 2024Large Language Models and Games: A Survey and Roadmap28 Feb 2024
- 2024Large Language Models as Minecraft Agents13 Feb 2024
- 2024Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue10 Feb 2024
- 2024Using Natural Language Inference to Improve Persona Extraction from Dialogue in a New Domain12 Jan 2024
- 2024Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talk10 Jan 2024
- 2023Chatbot is Not All You Need: Information-rich Prompting for More Realistic Responses25 Dec 2023
- 2023LARP: Language-Agent Role Play for Open-World Games24 Dec 2023
- 2023Constructing Temporal Dynamic Knowledge Graphs from Interactive Text-based Games3 Nov 2023
50 newest of 122 All 122 (JSON)
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
- 2023Humanoid Agents: Platform for Simulating Human-like Generative Agentscited by 73
- 2023Multi-Party Chat: Conversational Agents in Group Settings with Humans and Modelscited by 53
- 2024IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script Generationcited by 35
- 2024Bootstrapping LLM-based Task-Oriented Dialogue Agents via Self-Talkcited by 34
- 2023LARP: Language-Agent Role Play for Open-World Gamescited by 24
- 2024Security of AI Agentscited by 24
- 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
- 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordiacited by 11
- 2024MINDECHO: Role-Playing Language Agents for Key Opinion Leaderscited by 10
12 most cited of 20 All 20 (JSON)
- 2026TRACE Bench: Task-driven Roleplay Agentic Checklist Evaluation31 Jul 2026
- 2026NARRA-Gym for Evaluating Interactive Narrative Agents8 May 2026
- 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia3 Dec 2025
- 2025debug-gym: A Text-Based Environment for Interactive Debugging27 Mar 2025
- 2025Digital Player: Evaluating Large Language Models based Human-like Agent in Games28 Feb 2025
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmap13 Jan 2025
- 2024EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents18 Dec 2024
- 2024Drama Engine: A Framework for Narrative Agents21 Aug 2024
- 2024A LLM Benchmark based on the Minecraft Builder Dialog Agent Task17 Jul 2024
- 2024MINDECHO: Role-Playing Language Agents for Key Opinion Leaders7 Jul 2024
- 2024IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script Generation1 Jul 2024
- 2024Security of AI Agents12 Jun 2024
12 newest of 20 All 20 (JSON)
- 2026World Agent: Can Language Models Keep a World Running?26 Sep 2026
- 2023ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Games24 May 2023
- 2023Infusing Commonsense World Models with Graph Knowledge13 Jan 2023
- 2021Modeling Worlds in Text17 Jun 2021
- 2025SAGE: A Generic Framework for LLM Safety Evaluationcited by 7
- 2025SAGE: A Generic Framework for LLM Safety Evaluation28 Apr 2025
- 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
- 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
- 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
- 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
- 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
- 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
- 2024Large Language Models and Games: A Survey and Roadmapcited by 91
- 2022Evaluating Human-Language Model Interactioncited by 90
- 2023MindAgent: Emergent Gaming Interactioncited by 87
- 2021Beyond Fair Pay: Ethical Implications of NLP Crowdsourcingcited by 52
- 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 52
- 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
12 most cited of 90 All 90 (JSON)
- 2026Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse11 May 2026
- 2026Data Selection for Multi-turn Dialogue Instruction Tuning9 Apr 2026
- 2026Beyond Fixed Psychological Personas: State Beats Trait, but Language Models are State-Blind21 Jan 2026
- 2025GenQuest: An LLM-based Text Adventure Game for Language Learners6 Oct 2025
- 2025Can LLMs Generate High-Quality Task-Specific Conversations?4 Aug 2025
- 2025MDC-R: The Minecraft Dialogue Corpus with Reference27 Jun 2025
- 2025STORY2GAME: Generating (Almost) Everything in an Interactive Fiction Game6 May 2025
- 2025Word2Minecraft: Generating 3D Game Levels through Large Language Models18 Mar 2025
- 2025OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs14 Mar 2025
- 2024Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey14 Nov 2024
- 2024MCPDial: A Minecraft Persona-driven Dialogue Dataset29 Oct 2024
- 2024The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends21 Sep 2024
12 newest of 90 All 90 (JSON)