mlsanityData to 7 Oct 2026

A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges

arXiv 2403.10249 ↗First posted 15 Mar 2024cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

The swift evolution of Large-scale Models (LMs), either language-focused or multi-modal, has garnered extensive attention in both academy and industry. But despite the surge in interest in this rapidly evolving area, there are scarce systematic reviews on their capabilities and potential in distinct impactful scenarios. This paper endeavours to help bridge this gap, offering a thorough examination of the current landscape of LM usage in regards to complex game playing scenarios and the challenges still open. Here, we seek to systematically review the existing architectures of LM-based Agents (LMAs) for games and summarize their commonalities, challenges, and any other insights. Furthermore, we present our perspective on promising future research avenues for the advancement of LMs in games. We hope to assist researchers in gaining a clear understanding of the field and to generate more interest in this highly impactful research direction. A corresponding resource, continuously updated, can be found in our GitHub repository.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 72 of its 91 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

72 of 91 references.

References · 72

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  10. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  12. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
38 more
  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  2. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  3. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  4. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  5. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  6. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  7. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  8. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  9. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  10. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  11. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  12. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  13. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
  14. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  15. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
  16. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  17. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  18. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  19. 2023Agents: An Open-source Framework for Autonomous Language Agentscited by 116
  20. 2024Cradle: Empowering Foundation Agents Towards General Computer Controlcited by 114
  21. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  22. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
  23. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  24. 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 94
  25. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  26. 2023Embodied Task Planning with Large Language Modelscited by 84
  27. 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approachcited by 79
  28. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  29. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4cited by 67
  30. 2024Scaling Instructable Agents Across Many Simulated Worldscited by 67
  31. 2023MarioGPT: Open-Ended Text2Level Generation through Large Language Modelscited by 63
  32. 2023LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplaycited by 62
  33. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  34. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  35. 2023Lyfe Agents: Generative agents for low-cost real-time social interactionscited by 49
  36. 2023LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordinationcited by 48
  37. 2023Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worldscited by 45
  38. 2023GROOT: Learning to Follow Instructions by Watching Gameplay Videoscited by 42

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 8; 2026: 3.

22 papers.

Sort:

By area:

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  2. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
  3. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  4. 2024Odyssey: Empowering Minecraft Agents with Open-World Skillscited by 26
  5. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  6. 2024PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videoscited by 21
  7. 2024Surveying the MLLM Landscape: A Meta-Review of Current Surveyscited by 20
  8. 2025Are Large Vision Language Models Good Game Players?cited by 19
  9. 2024On Large Language Models in National Security Applicationscited by 18
  10. 2024StraGo: Harnessing Strategic Guidance for Prompt Optimizationcited by 16
  11. 2025DeepPHY: Benchmarking Agentic VLMs on Physical Reasoningcited by 13
  12. 2024VideoGameBunny: Towards vision assistants for video gamescited by 11
10 more
  1. 2024ING-VP: MLLMs cannot Play Easy Vision-based Games Yetcited by 10
  2. 2025Taking Notes Brings Focus? Towards Multi-Turn Multimodal Dialogue Learningcited by 5
  3. 2025Leveraging LLM Agents for Automated Video Game Testingcited by 4
  4. 2025Memory-Augmented State Machine Prompting: A Novel LLM Agent Framework for Real-Time Strategy Gamescited by 2
  5. 2026AI for Games in the Foundation Model Eracited by 2
  6. 2025Shall We Play a Game? Language Models for Open-ended Wargamescited by 1
  7. 2025Can They Dixit? Yes they Can! Dixit as a Playground for Multimodal Language Model Capabilitiescited by 1
  8. 2026Order from Chaos: Physical World Understanding from Glitchy Gameplay Videoscited by 1
  9. 2024Extract Information from Hybrid Long Documents Leveraging LLMs: A Framework and Datasetcited by 0
  10. 2026Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Playcited by 0