A Minimal Approach for Natural Language Action Space in Text-based Games
Not on the map. Why?
Abstract (arXiv)
Text-based games (TGs) are language-based interactive environments for reinforcement learning. While language models (LMs) and knowledge graphs (KGs) are commonly used for handling large action space in TGs, it is unclear whether these techniques are necessary or overused. In this paper, we revisit the challenge of exploring the action space in TGs and propose ε-admissible exploration, a minimal approach of utilizing admissible actions, for training phase. Additionally, we present a text-based actor-critic (TAC) agent that produces textual commands for game, solely from game observations, without requiring any KG or LM. Our method, on average across 10 games from Jericho, outperforms strong baselines and state-of-the-art agents that use LM and KG. Our approach highlights that a much lighter model design, with a fresh perspective on utilizing the information within the environments, suffices for an effective exploration of exponentially large action spaces.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 48 references that are in the map. First posted on arXiv 6 May 2023.
Builds on
21 of 48 references.
References · 21
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2015Prioritized Experience Replaycited by 1,070
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
- 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
- 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
- 2019Interactive Fiction Games: A Colossal Adventurecited by 165
- 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
- 2019Observational Overfitting in Reinforcement Learningcited by 90
- 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
9 more
- 2017Learning how to Active Learn: A Deep Reinforcement Learning Approachcited by 79
- 2020Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Gamescited by 22
- 2022Multi-Stage Episodic Control for Strategic Exploration in Text Gamescited by 22
- 2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
- 2021Modeling Worlds in Textcited by 11
- 2021Generalization in Text-based Games via Hierarchical Reinforcement Learningcited by 11
- 2020How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agentscited by 10
- 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10
- 2020Do not let the history haunt you -- Mitigating Compounding Errors in Conversational Question Answeringcited by 6
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.