mlsanityData to 14 Oct 2026

Monte Carlo Planning with Large Language Model for Text-Based Game Agents

arXiv 2504.16855 ↗First posted 23 Apr 2025cs.CLcited by 22

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Text-based games provide valuable environments for language-based autonomous agents. However, planning-then-learning paradigms, such as those combining Monte Carlo Tree Search (MCTS) and reinforcement learning (RL), are notably time-consuming due to extensive iterations. Additionally, these algorithms perform uncertainty-driven exploration but lack language understanding and reasoning abilities. In this paper, we introduce the Monte Carlo planning with Dynamic Memory-guided Large language model (MC-DML) algorithm. MC-DML leverages the language understanding and reasoning capabilities of Large Language Models (LLMs) alongside the exploratory advantages of tree search algorithms. Specifically, we enhance LLMs with in-trial and cross-trial memory mechanisms, enabling them to learn from past experiences and dynamically adjust action evaluations during planning. We conduct experiments on a series of text-based games from the Jericho benchmark. Our results demonstrate that the MC-DML algorithm significantly enhances performance across various games at the initial planning phase, outperforming strong contemporary methods that require multiple iterations. This demonstrates the effectiveness of our algorithm, paving the way for more efficient language-grounded planning in complex environments.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 34 of its 55 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

34 of 55 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agents14 Oct
  2. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory25 May
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar

Other references · 31

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  3. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  6. 2018TextWorld: A Learning Environment for Text-based Gamescited by 310
  7. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  8. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  9. 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 161
  10. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  11. 2015Deep Reinforcement Learning with a Natural Language Action Spacecited by 88
  12. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
19 more
  1. 2018Learn What Not to Learn: Action Elimination with Deep Reinforcement Learningcited by 77
  2. 2023Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generationcited by 74
  3. 2015Language Understanding for Text-based Games Using Deep Reinforcement Learningcited by 71
  4. 2024Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Taskscited by 52
  5. 2020Learning Dynamic Belief Graphs to Generalize on Text-Based Gamescited by 49
  6. 2014Learning to Win by Reading Manuals in a Monte-Carlo Frameworkcited by 39
  7. 2020How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worldscited by 38
  8. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  9. 2024Large Language Models Are Neurosymbolic Reasonerscited by 31
  10. 2020Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learningcited by 27
  11. 2020Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Gamescited by 22
  12. 2022Multi-Stage Episodic Control for Strategic Exploration in Text Gamescited by 22
  13. 2020Playing Text-Based Games with Common Sensecited by 21
  14. 2021A Survey of Text Games for Reinforcement Learning informed by Natural Languagecited by 14
  15. 2024How Can LLM Guide RL? A Value-Based Approachcited by 13
  16. 2021Generalization in Text-based Games via Hierarchical Reinforcement Learningcited by 11
  17. 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10
  18. 2021Case-based Reasoning for Better Generalization in Textual Reinforcement Learningcited by 8
  19. 2021Simplified Belief-Dependent Reward MCTS Planning with Guaranteed Tree Consistencycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 17.

22 papers.

Sort:

By area:

  1. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  2. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
  3. 2026LongSeeker: Elastic Context Orchestration for Long-Horizon Search Agentscited by 10
  4. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 8
  5. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 5
  6. 2026SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assemblycited by 5
  7. 2025Co-Saving: Resource Aware Multi-Agent Collaboration for Software Developmentcited by 3
  8. 2026Benchmarking Web Agent Safety under E-commerce Deceptive Interfacescited by 3
  9. 2026GraphPO: Graph-based Policy Optimization for Reasoning Modelscited by 3
  10. 2025Dual-Scale World Models for LLM Agents Towards Hard-Exploration Problemscited by 2
  11. 2026Tree-Structured Synergy of Large Language Models and Bayesian Optimization for Efficient CASHcited by 2
  12. 2025ToC: Tree-of-Claims Search with Multi-Agent Language Modelscited by 1
10 more
  1. 2026Task-Aware LLM Council with Adaptive Decision Pathways for Decision Supportcited by 1
  2. 2026UIPress: Bringing Optical Token Compression to UI-to-Code Generationcited by 1
  3. 2026BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Designcited by 1
  4. 2026PriorZero: Bridging Language Priors and World Models for Decision Makingcited by 1
  5. 2026Adaptive Parallel Monte Carlo Tree Search for Efficient Test-time Compute Scalingcited by 0
  6. 2026Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futurescited by 0
  7. 2026Particle Monte Carlo Tree Searchcited by 0
  8. 2026Joint Agent Memory and Exploration Learning via Novelty Signalscited by 0
  9. 2026GUI agent: Guided Exploration of User-Sensitive Screenscited by 0
  10. 2026SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videoscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Can Large Language Models Play Games? A Case Study of A Self-Play Approach
  2. 2025MASTER: A Multi-Agent System with LLM Specialized MCTS
  3. 2024Large Language Models as Agents in Two-Player Games
  4. 2024On the Modeling Capabilities of Large Language Models for Sequential Decision Making
  5. 2024Mastering Board Games by External and Internal Planning with Language Models
  6. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods
  7. 2024Query-Efficient Planning with Language Models
  8. 2025Toward Efficient Exploration by Large Language Model Agents