mlsanityData to 14 Oct 2026

Generalization in Text-based Games via Hierarchical Reinforcement Learning

arXiv 2109.09968 ↗First posted 21 Sep 2021cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning provides a promising approach for text-based games in studying natural language communication between humans and artificial agents. However, the generalization still remains a big challenge as the agents depend critically on the complexity and variety of training tasks. In this paper, we address this problem by introducing a hierarchical framework built upon the knowledge graph-based RL agent. In the high level, a meta-policy is executed to decompose the whole game into a set of subtasks specified by textual goals, and select one of them based on the KG. Then a sub-policy in the low level is executed to conduct goal-conditioned reinforcement learning. We carry out experiments on games with various difficulty levels and show that the proposed method enjoys favorable generalizability.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 49 of its 59 references that are in the map. First posted on arXiv 21 Sep 2021.

Builds on

49 of 59 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  4. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2015Prioritized Experience Replaycited by 1,099
  7. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  9. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  10. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  11. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
  12. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
37 more
  1. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  2. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  3. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  4. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  5. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  6. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  7. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 98
  8. 2015Deep Reinforcement Learning with a Natural Language Action Spacecited by 88
  9. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  10. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  11. 2017Learning how to Active Learn: A Deep Reinforcement Learning Approachcited by 79
  12. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  13. 2018Learn What Not to Learn: Action Elimination with Deep Reinforcement Learningcited by 77
  14. 2015Language Understanding for Text-based Games Using Deep Reinforcement Learningcited by 71
  15. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  16. 2018TACO: Learning Task Decomposition via Temporal Alignment for Controlcited by 62
  17. 2018Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learningcited by 53
  18. 2020Learning Dynamic Belief Graphs to Generalize on Text-Based Gamescited by 49
  19. 2017Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learningcited by 44
  20. 2018Hierarchical Reinforcement Learning for Zero-shot Generalization with Subtask Dependenciescited by 43
  21. 2020Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselinescited by 37
  22. 2018Counting to Explore and Generalize in Text-based Gamescited by 36
  23. 2020How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worldscited by 35
  24. 2020BeBold: Exploration Beyond the Boundary of Explored Regionscited by 33
  25. 2019Hierarchical Reinforcement Learning for Open-Domain Dialogcited by 27
  26. 2020Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learningcited by 27
  27. 2020Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Gamescited by 22
  28. 2019NAIL: A General Interactive Fiction Agentcited by 17
  29. 2019LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Gamescited by 17
  30. 2020Enhancing Text-based Reinforcement Learning Agents with Commonsense Knowledgecited by 16
  31. 2019Toward Automated Quest Generation in Text-Adventure Gamescited by 14
  32. 2019Algorithmic Improvements for Deep Reinforcement Learning applied to Interactive Fictioncited by 14
  33. 2019Comprehensible Context-driven Text Game Playingcited by 10
  34. 2020How To Avoid Being Eaten By a Grue: Exploration Strategies for Text-Adventure Agentscited by 10
  35. 2018The Text-Based Adventure AI Competitioncited by 5
  36. 2019Transfer in Deep Reinforcement Learning using Knowledge Graphscited by 5
  37. 2020Learning to Generalize for Sequential Decision Makingcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 3; 2024: 1; 2025: 4.

11 papers.

  1. 2023Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Gamecited by 28
  2. 2022Multi-Stage Episodic Control for Strategic Exploration in Text Gamescited by 22
  3. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  4. 2024Player-Driven Emergence in LLM-Driven Game Narrativecited by 16
  5. 2022TextWorldExpress: Simulating Text Games at One Million Steps Per Secondcited by 13
  6. 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10
  7. 2023Knowledge-enhanced Agents for Interactive Text Gamescited by 10
  8. 2025Divide-Fuse-Conquer: Eliciting "Aha Moments" in Multi-Scenario Gamescited by 4
  9. 2023A Minimal Approach for Natural Language Action Space in Text-based Gamescited by 3
  10. 2025Learning Natural Language Constraints for Safe Reinforcement Learning of Language Agentscited by 2
  11. 2025Persona Dynamics: Unveiling the Impact of Personality Traits on Agents in Text-Based Gamescited by 1