mlsanityData to 13 Oct 2026

Learning Dynamic Belief Graphs to Generalize on Text-Based Games

arXiv 2002.09127 ↗First posted 21 Feb 2020cs.CL cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

Playing text-based games requires skills in processing natural language and sequential decision making. Achieving human-level performance on text-based games remains an open challenge, and prior research has largely relied on hand-crafted structured representations and heuristics. In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text. We propose a novel graph-aided transformer agent (GATA) that infers and updates latent belief graphs during planning to enable effective action selection by capturing the underlying game dynamics. GATA is trained using a combination of reinforcement and self-supervised learning. Our work demonstrates that the learned graph-based representations help agents converge to better policies than their text-only counterparts and facilitate effective generalization across game configurations. Experiments on 500+ unique games from the TextWorld suite show that our best agent outperforms text-based baselines by an average of 24.2%.

Key facts. Cited by 49 papers in the map (data through 13 Oct 2026). First posted on arXiv 21 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 14; 2022: 11; 2023: 8; 2024: 6; 2025: 7; 2026: 3.

49 papers.

Sort:

By area:

  1. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 250
  2. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 177
  3. 2024AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agentscited by 102
  4. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  5. 2021What Would Jiminy Cricket Do? Towards Agents That Behave Morallycited by 57
  6. 2024Can Language Models Serve as Text-Based World Simulators?cited by 51
  7. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  8. 2022Aligning to Social Norms and Values in Interactive Narrativescited by 32
  9. 2021Learning Knowledge Graph-based World Models of Textual Environmentscited by 31
  10. 2021Neuro-Symbolic Reinforcement Learning with First-Order Logiccited by 30
  11. 2023Introspective Tips: Large Language Model for In-Context Decision Makingcited by 28
  12. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
37 more
  1. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  2. 2021A Systematic Survey of Text Worlds as Embodied Natural Language Environmentscited by 17
  3. 2021Process-Level Representation of Scientific Protocols with Interactive Annotationcited by 16
  4. 2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
  5. 2021Towards Socially Intelligent Agents with Mental State Transition and Human Utilitycited by 14
  6. 2021A Survey of Text Games for Reinforcement Learning informed by Natural Languagecited by 14
  7. 2022Learning to Follow Instructions in Text-Based Gamescited by 12
  8. 2021Modeling Worlds in Textcited by 11
  9. 2021How to Query Language Models?cited by 11
  10. 2021Generalization in Text-based Games via Hierarchical Reinforcement Learningcited by 11
  11. 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10
  12. 2022Asking for Knowledge: Training RL Agents to Query External Knowledge Using Languagecited by 10
  13. 2022Behavior Cloned Transformers are Neurosymbolic Reasonerscited by 10
  14. 2022Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learningcited by 10
  15. 2021Situated Dialogue Learning through Procedural Environment Generationcited by 9
  16. 2024Can LLMs perform structured graph reasoning?cited by 9
  17. 2025GraphThought: Graph Combinatorial Optimization with Thought Generationcited by 9
  18. 2021LOA: Logical Optimal Actions for Text-based Interaction Gamescited by 6
  19. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 5
  20. 2022Building Human-like Communicative Intelligence: A Grounded Perspectivecited by 4
  21. 2023Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learningcited by 4
  22. 2025CoEx -- Co-evolving World-model and Explorationcited by 4
  23. 2024EXPLORER: Exploration-guided Reasoning for Textual Reinforcement Learningcited by 3
  24. 2021Interactive Machine Comprehension with Dynamic Knowledge Graphscited by 2
  25. 2023ScriptWorld: Text Based Environment For Learning Procedural Knowledgecited by 2
  26. 2024A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worldscited by 2
  27. 2025Learning Natural Language Constraints for Safe Reinforcement Learning of Language Agentscited by 2
  28. 2022Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learningcited by 1
  29. 2022An Analysis of Deep Reinforcement Learning Agents for Text-based Gamescited by 1
  30. 2023Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphscited by 1
  31. 2023Constructing Temporal Dynamic Knowledge Graphs from Interactive Text-based Gamescited by 1
  32. 2025Efficient Graph Understanding with LLMs via Structured Context Injectioncited by 1
  33. 2026Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluationcited by 1
  34. 2026Decision-Sufficient State Representations: Measuring and Reducing Write-Time Regretcited by 1
  35. 2022DiffG-RL: Leveraging Difference between State and Common Sensecited by 0
  36. 2024Can Language Models Enable In-Context Database?cited by 0
  37. 2026Textual Planning with Explicit Latent Transitionscited by 0