# A Survey of Text Games for Reinforcement Learning informed by Natural Language

arXiv: https://arxiv.org/abs/2109.09478 · HTML: https://mlsanity.com/paper/2109.09478/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 53 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [TextWorld: A Learning Environment for Text-based Games](https://mlsanity.com/paper/1806.11532/) (2018)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Interactive Fiction Games: A Colossal Adventure](https://mlsanity.com/paper/1909.05398/) (2019)
- [A Survey of Reinforcement Learning Informed by Natural Language](https://mlsanity.com/paper/1906.03926/) (2019)
- [Learning to Speak and Act in a Fantasy Text Adventure Game](https://mlsanity.com/paper/1903.03094/) (2019)
- [Keep CALM and Explore: Language Models for Action Generation in Text-based Games](https://mlsanity.com/paper/2010.02903/) (2020)
- [Deep Reinforcement Learning with a Natural Language Action Space](https://mlsanity.com/paper/1511.04636/) (2015)
- [Graph Constrained Reinforcement Learning for Natural Language Action Spaces](https://mlsanity.com/paper/2001.08837/) (2020)
- [Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning](https://mlsanity.com/paper/1809.02121/) (2018)
- [Language Understanding for Text-based Games Using Deep Reinforcement Learning](https://mlsanity.com/paper/1506.08941/) (2015)
- [Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning](https://mlsanity.com/paper/1812.01628/) (2018)
- [Learning Dynamic Belief Graphs to Generalize on Text-Based Games](https://mlsanity.com/paper/2002.09127/) (2020)
- [How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worlds](https://mlsanity.com/paper/2006.07409/) (2020)
- [Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines](https://mlsanity.com/paper/2010.03790/) (2020)
- [Counting to Explore and Generalize in Text-based Games](https://mlsanity.com/paper/1806.11525/) (2018)
- [How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds](https://mlsanity.com/paper/2010.00685/) (2020)
- [Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning](https://mlsanity.com/paper/2010.02386/) (2020)
- [Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games](https://mlsanity.com/paper/2010.11655/) (2020)
- [Interactive Language Learning by Question Answering](https://mlsanity.com/paper/1908.10909/) (2019)
- [Exploration Based Language Learning for Text-Based Games](https://mlsanity.com/paper/2001.08868/) (2020)
- [Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension](https://mlsanity.com/paper/1810.05682/) (2018)
- [NAIL: A General Interactive Fiction Agent](https://mlsanity.com/paper/1902.04259/) (2019)

## Built on by (most cited first, 14 of 14)

- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Introspective Tips: Large Language Model for In-Context Decision Making](https://mlsanity.com/paper/2305.11598/) (2023)
- [Multi-Stage Episodic Control for Strategic Exploration in Text Games](https://mlsanity.com/paper/2201.01251/) (2022)
- [Monte Carlo Planning with Large Language Model for Text-Based Game Agents](https://mlsanity.com/paper/2504.16855/) (2025)
- [TextWorldExpress: Simulating Text Games at One Million Steps Per Second](https://mlsanity.com/paper/2208.01174/) (2022)
- [TALES: Text Adventure Learning Environment Suite](https://mlsanity.com/paper/2504.14128/) (2025)
- [Behavior Cloned Transformers are Neurosymbolic Reasoners](https://mlsanity.com/paper/2210.07382/) (2022)
- [Conceptual Reinforcement Learning for Language-Conditioned Tasks](https://mlsanity.com/paper/2303.05069/) (2023)
- [Digital Player: Evaluating Large Language Models based Human-like Agent in Games](https://mlsanity.com/paper/2502.20807/) (2025)
- [A Generalist Hanabi Agent](https://mlsanity.com/paper/2503.14555/) (2025)
- [A Survey on Complex Tasks for Goal-Directed Interactive Agents](https://mlsanity.com/paper/2409.18538/) (2024)
- [Learning Natural Language Constraints for Safe Reinforcement Learning of Language Agents](https://mlsanity.com/paper/2504.03185/) (2025)
- [Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation](https://mlsanity.com/paper/2604.11611/) (2026)
- [elsciRL: Integrating Language Solutions into Reinforcement Learning Problem Settings](https://mlsanity.com/paper/2507.08705/) (2025)

## Abstract (arXiv)

Reinforcement Learning has shown success in a number of complex virtual environments. However, many challenges still exist towards solving problems with natural language as a core component. Interactive Fiction Games (or Text Games) are one such problem type that offer a set of partially observable environments where natural language is required as part of the reinforcement learning solutions. Therefore, this survey's aim is to assist in the development of new Text Game problem settings and solutions for Reinforcement Learning informed by natural language. Specifically, this survey summarises: 1) the challenges introduced in Text Game Reinforcement Learning problems, 2) the generation tools for evaluating Text Games and the subsequent environments generated and, 3) the agent architectures currently applied are compared to provide a systematic review of benchmark methodologies and opportunities for future researchers.