# Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines

arXiv: https://arxiv.org/abs/2010.03790 · HTML: https://mlsanity.com/paper/2010.03790/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 30 of 37)

- [ScienceWorld: Is your Agent Smarter than a 5th Grader?](https://mlsanity.com/paper/2203.07540/) (2022)
- [SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks](https://mlsanity.com/paper/2305.17390/) (2023)
- [AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents](https://mlsanity.com/paper/2407.04363/) (2024)
- [Neuro-Symbolic AI in 2024: A Systematic Review](https://mlsanity.com/paper/2501.05435/) (2025)
- [Aligning to Social Norms and Values in Interactive Narratives](https://mlsanity.com/paper/2205.01975/) (2022)
- [Learning Knowledge Graph-based World Models of Textual Environments](https://mlsanity.com/paper/2106.09608/) (2021)
- [Neuro-Symbolic Reinforcement Learning with First-Order Logic](https://mlsanity.com/paper/2110.10963/) (2021)
- [A Survey of Text Games for Reinforcement Learning informed by Natural Language](https://mlsanity.com/paper/2109.09478/) (2021)
- [TextWorldExpress: Simulating Text Games at One Million Steps Per Second](https://mlsanity.com/paper/2208.01174/) (2022)
- [Modeling Worlds in Text](https://mlsanity.com/paper/2106.09578/) (2021)
- [How to Query Language Models?](https://mlsanity.com/paper/2108.01928/) (2021)
- [Generalization in Text-based Games via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2109.09968/) (2021)
- [World of Workflows: A Benchmark for Bringing World Models to Enterprise Systems](https://mlsanity.com/paper/2601.22130/) (2026)
- [Perceiving the World: Question-guided Reinforcement Learning for Text-based Games](https://mlsanity.com/paper/2204.09597/) (2022)
- [Behavior Cloned Transformers are Neurosymbolic Reasoners](https://mlsanity.com/paper/2210.07382/) (2022)
- [Situated Dialogue Learning through Procedural Environment Generation](https://mlsanity.com/paper/2110.03262/) (2021)
- [Language Guided Exploration for RL Agents in Text Environments](https://mlsanity.com/paper/2403.03141/) (2024)
- [Case-based Reasoning for Better Generalization in Textual Reinforcement Learning](https://mlsanity.com/paper/2110.08470/) (2021)
- [Conceptual Reinforcement Learning for Language-Conditioned Tasks](https://mlsanity.com/paper/2303.05069/) (2023)
- [STARLING: Self-supervised Training of Text-based Reinforcement Learning Agent with Large Language Models](https://mlsanity.com/paper/2406.05872/) (2024)
- [LOA: Logical Optimal Actions for Text-based Interaction Games](https://mlsanity.com/paper/2110.10973/) (2021)
- [Eye of the Beholder: Improved Relation Generalization for Text-based Reinforcement Learning Agents](https://mlsanity.com/paper/2106.05387/) (2021)
- [Infusing Lattice Symmetry Priors in Attention Mechanisms for Sample-Efficient Abstract Geometric Reasoning](https://mlsanity.com/paper/2306.03175/) (2023)
- [Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning](https://mlsanity.com/paper/2307.02689/) (2023)
- [Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting](https://mlsanity.com/paper/2410.19920/) (2024)
- [EXPLORER: Exploration-guided Reasoning for Textual Reinforcement Learning](https://mlsanity.com/paper/2403.10692/) (2024)
- [Leveraging class abstraction for commonsense reinforcement learning via residual policy gradient methods](https://mlsanity.com/paper/2201.12126/) (2022)
- [Polar Ducks and Where to Find Them: Enhancing Entity Linking with Duck Typing and Polar Box Embeddings](https://mlsanity.com/paper/2305.12027/) (2023)
- [ScriptWorld: Text Based Environment For Learning Procedural Knowledge](https://mlsanity.com/paper/2307.03906/) (2023)
- [ActionCOMET: A Zero-shot Approach to Learn Image-specific Commonsense Concepts about Actions](https://mlsanity.com/paper/2410.13662/) (2024)

## Abstract (arXiv)

Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making. In this paper, we examine the problem of infusing RL agents with commonsense knowledge. Such knowledge would allow agents to efficiently act in the world by pruning out implausible actions, and to perform look-ahead planning to determine how current actions might affect future world states. We design a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances. We also introduce several baseline RL agents which track the sequential context and dynamically retrieve the relevant commonsense knowledge from ConceptNet. We show that agents which incorporate commonsense knowledge in TWC perform better, while acting more efficiently. We conduct user-studies to estimate human performance on TWC and show that there is ample room for future improvement.