# Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games

arXiv: https://arxiv.org/abs/2010.11655 · HTML: https://mlsanity.com/paper/2010.11655/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2020.

## Built on by (most cited first, 22 of 22)

- [Hierarchical Learning for Generation with Long Source Sequences](https://mlsanity.com/paper/2104.07545/) (2021)
- [DAGN: Discourse-Aware Graph Network for Logical Reasoning](https://mlsanity.com/paper/2103.14349/) (2021)
- [Cooperation on the Fly: Exploring Language Agents for Ad Hoc Teamwork in the Avalon Game](https://mlsanity.com/paper/2312.17515/) (2023)
- [Multi-Stage Episodic Control for Strategic Exploration in Text Games](https://mlsanity.com/paper/2201.01251/) (2022)
- [Monte Carlo Planning with Large Language Model for Text-Based Game Agents](https://mlsanity.com/paper/2504.16855/) (2025)
- [Pre-trained Language Models as Prior Knowledge for Playing Text-based Games](https://mlsanity.com/paper/2107.08408/) (2021)
- [A Systematic Survey of Text Worlds as Embodied Natural Language Environments](https://mlsanity.com/paper/2107.04132/) (2021)
- [Inherently Explainable Reinforcement Learning in Natural Language](https://mlsanity.com/paper/2112.08907/) (2021)
- [Training Value-Aligned Reinforcement Learning Agents Using a Normative Prior](https://mlsanity.com/paper/2104.09469/) (2021)
- [A Survey of Text Games for Reinforcement Learning informed by Natural Language](https://mlsanity.com/paper/2109.09478/) (2021)
- [Generalization in Text-based Games via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2109.09968/) (2021)
- [Perceiving the World: Question-guided Reinforcement Learning for Text-based Games](https://mlsanity.com/paper/2204.09597/) (2022)
- [Behavior Cloned Transformers are Neurosymbolic Reasoners](https://mlsanity.com/paper/2210.07382/) (2022)
- [Multi-Turn Puzzles: Evaluating Interactive Reasoning and Strategic Dialogue in LLMs](https://mlsanity.com/paper/2508.10142/) (2025)
- [Dialogue Shaping: Empowering Agents through NPC Interaction](https://mlsanity.com/paper/2307.15833/) (2023)
- [Story Shaping: Teaching Agents Human-like Behavior with Stories](https://mlsanity.com/paper/2301.10107/) (2023)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Language Decision Transformers with Exponential Tilt for Interactive Text Environments](https://mlsanity.com/paper/2302.05507/) (2023)
- [A Minimal Approach for Natural Language Action Space in Text-based Games](https://mlsanity.com/paper/2305.04082/) (2023)
- [Learning to Play Text-based Adventure Games with Maximum Entropy Reinforcement Learning](https://mlsanity.com/paper/2302.10720/) (2023)
- [An Analysis of Deep Reinforcement Learning Agents for Text-based Games](https://mlsanity.com/paper/2209.04105/) (2022)
- [Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games](https://mlsanity.com/paper/2312.04657/) (2023)

## Abstract (arXiv)

We study reinforcement learning (RL) for text-based games, which are interactive simulations in the context of natural language. While different methods have been developed to represent the environment information and language actions, existing RL agents are not empowered with any reasoning capabilities to deal with textual games. In this work, we aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure. We propose a stacked hierarchical attention mechanism to construct an explicit representation of the reasoning process by exploiting the structure of the knowledge graph. We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.