mlsanityData to 14 Oct 2026

Perceiving the World: Question-guided Reinforcement Learning for Text-based Games

arXiv 2204.09597 ↗First posted 20 Mar 2022cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Text-based games provide an interactive way to study natural language processing. While deep reinforcement learning has shown effectiveness in developing the game playing agent, the low sample efficiency and the large action space remain to be the two major challenges that hinder the DRL from being applied in the real world. In this paper, we address the challenges by introducing world-perceiving modules, which automatically decompose tasks and prune actions by answering questions about the environment. We then propose a two-phase training framework to decouple language learning from reinforcement learning, which further improves the sample efficiency. The experimental results show that the proposed method significantly improves the performance and sample efficiency. Besides, it shows robustness against compound error and limited pre-training data.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 59 of its 72 references that are in the map. First posted on arXiv 20 Mar 2022.

Builds on

59 of 72 references.

References · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  3. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  6. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  7. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
  8. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  9. 2017Advances in Pre-Training Distributed Word Representationscited by 207
  10. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  11. 2020Data Augmentation for Graph Neural Networkscited by 203
  12. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
38 more
  1. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  2. 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 161
  3. 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 140
  4. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  5. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 133
  6. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  7. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  8. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  9. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  10. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  11. 2015Deep Reinforcement Learning with a Natural Language Action Spacecited by 88
  12. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 88
  13. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  14. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  15. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  16. 2017Learning how to Active Learn: A Deep Reinforcement Learning Approachcited by 79
  17. 2018Learn What Not to Learn: Action Elimination with Deep Reinforcement Learningcited by 77
  18. 2019Learning Action Representations for Reinforcement Learningcited by 75
  19. 2015Language Understanding for Text-based Games Using Deep Reinforcement Learningcited by 71
  20. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  21. 2018TACO: Learning Task Decomposition via Temporal Alignment for Controlcited by 62
  22. 2020Grounded Language Learning Fast and Slowcited by 54
  23. 2018Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learningcited by 53
  24. 2020Learning Dynamic Belief Graphs to Generalize on Text-Based Gamescited by 49
  25. 2018Hierarchical Reinforcement Learning for Zero-shot Generalization with Subtask Dependenciescited by 43
  26. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  27. 2018Investigating Human Priors for Playing Video Gamescited by 39
  28. 2021Hierarchical Skills for Efficient Explorationcited by 38
  29. 2020Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselinescited by 37
  30. 2018Counting to Explore and Generalize in Text-based Gamescited by 36
  31. 2020How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worldscited by 35
  32. 2020Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learningcited by 32
  33. 2019Hierarchical Decision Making by Generating and Following Natural Language Instructionscited by 30
  34. 2020Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learningcited by 27
  35. 2020Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Gamescited by 22
  36. 2021Pre-trained Language Models as Prior Knowledge for Playing Text-based Gamescited by 21
  37. 2017Accurately and Efficiently Interpreting Human-Robot Instructions of Varying Granularitiescited by 19
  38. 2017Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Managementcited by 18

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 4.

10 papers.

  1. 2023Introspective Tips: Large Language Model for In-Context Decision Makingcited by 28
  2. 2023Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Gamecited by 28
  3. 2022EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RLcited by 23
  4. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  5. 2024Player-Driven Emergence in LLM-Driven Game Narrativecited by 16
  6. 2025Learning to Play Like Humans: A Framework for LLM Adaptation in Interactive Fiction Gamescited by 4
  7. 2022A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challengescited by 3
  8. 2023A Minimal Approach for Natural Language Action Space in Text-based Gamescited by 3
  9. 2025Persona Dynamics: Unveiling the Impact of Personality Traits on Agents in Text-Based Gamescited by 1
  10. 2025A Convolution and Attention Based Encoder for Reinforcement Learning under Partial Observabilitycited by 0