mlsanityData to 7 Oct 2026

Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questions

arXiv 2304.02868 ↗First posted 6 Apr 2023cs.CL cs.AI cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) such as ChatGPT and GPT-4 have recently demonstrated their remarkable abilities of communicating with human users. In this technical report, we take an initiative to investigate their capacities of playing text games, in which a player has to understand the environment and respond to situations by having dialogues with the game world. Our experiments show that ChatGPT performs competitively compared to all the existing systems but still exhibits a low level of intelligence. Precisely, ChatGPT can not construct the world model by playing the game or even reading the game manual; it may fail to leverage the world knowledge that it already has; it cannot infer the goal of each step as the game progresses. Our results open up new research questions at the intersection of artificial intelligence, machine learning, and natural language processing.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

11 of 20 references.

References · 11

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  6. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 228
  7. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  8. 2020Graph Constrained Reinforcement Learning for Natural Language Action Spacescited by 80
  9. 2019AlphaStar: An Evolutionary Computation Perspectivecited by 50
  10. 2020Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learningcited by 27
  11. 2019NAIL: A General Interactive Fiction Agentcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 14; 2025: 7; 2026: 2.

30 papers.

Sort:

By area:

  1. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  2. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
  3. 2024Large Language Models and Games: A Survey and Roadmapcited by 91
  4. 2024How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environmentscited by 66
  5. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 62
  6. 2023Entity Tracking in Language Modelscited by 44
  7. 2024Do LLM Agents Have Regret? A Case Study in Online Learning and Gamescited by 33
  8. 2025Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Gamescited by 30
  9. 2025G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learningcited by 29
  10. 2024A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challengescited by 22
  11. 2024Evaluating Large Language Models with Grid-Based Game Competitions: An Extensible LLM Benchmark and Leaderboardcited by 22
  12. 2023Are ChatGPT and GPT-4 Good Poker Players? -- A Pre-Flop Analysiscited by 20
18 more
  1. 2024Player-Driven Emergence in LLM-Driven Game Narrativecited by 16
  2. 2024Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineeringcited by 16
  3. 2023Assessing Logical Puzzle Solving in Large Language Models: Insights from a Minesweeper Case Studycited by 14
  4. 2023SPRING: Studying the Paper and Reasoning to Play Gamescited by 11
  5. 2024How Well Can Vision Language Models See Image Details?cited by 10
  6. 2024Missed Connections: Lateral Thinking Puzzles for Large Language Modelscited by 9
  7. 2025Agents Play Thousands of 3D Video Gamescited by 9
  8. 2025TALES: Text Adventure Learning Environment Suitecited by 9
  9. 2023Chain-of-Symbol Prompting Elicits Planning in Large Langauge Modelscited by 6
  10. 2023Exploring the Limits of ChatGPT in Software Security Applicationscited by 5
  11. 2025Tracing LLM Reasoning Processes with Strategic Games: A Framework for Planning, Revision, and Resource-Constrained Decision Makingcited by 5
  12. 2024Take A Step Back: Rethinking the Two Stages in Visual Reasoningcited by 4
  13. 2025Reinforcement Learning Environment with LLM-Controlled Adversary in D&D 5th Edition Combatcited by 3
  14. 2025Can Large Language Models Capture Video Game Engagement?cited by 2
  15. 2024A Text-to-Game Engine for UGC-Based Role-Playing Gamescited by 1
  16. 2024Malinowski in the Age of AI: Can large language models create a text game based on an anthropological classic?cited by 1
  17. 2026Playing DOOM with 1.3M Parameters: Specialized Small Models vs Large Language Models for Real-Time Game Controlcited by 1
  18. 2026High-quality generation of dynamic game content via small language models: A proof of conceptcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Playing With AI: How Do State-Of-The-Art Large Language Models Perform in the 1977 Text-Based Adventure Game Zork?
  2. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  3. 2023Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents
  4. 2025TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning
  5. 2024How Many Parameters Does it Take to Change a Light Bulb? Evaluating Performance in Self-Play of Conversational Games as a Function of Model Characteristics
  6. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
  7. 2024clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents
  8. 2024GameArena: Evaluating LLM Reasoning through Live Computer Games