Will GPT-4 Run DOOM?
Not on the map. Why?
Abstract (arXiv)
We show that GPT-4's reasoning and planning capabilities extend to the 1993 first-person shooter Doom. This large language model (LLM) is able to run and play the game with only a few instructions, plus a textual description--generated by the model itself from screenshots--about the state of the game being observed. We find that GPT-4 can play the game to a passable degree: it is able to manipulate doors, combat enemies, and perform pathing. More complex prompting strategies involving multiple model calls provide better results. While further work is required to enable the LLM to play the game as well as its classical, reinforcement learning-based counterparts, we note that GPT-4 required no training, leaning instead on its own reasoning and observational capabilities. We hope our work pushes the boundaries on intelligent, LLM-based agents in video games. We conclude by discussing the ethical implications of our work.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 39 of its 55 references that are in the map. First posted on arXiv 8 Mar 2024.
Builds on
39 of 55 references.
References · 39
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
- 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
- 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
- 2023Reasoning with Language Model is Planning with World Modelcited by 846
- 2022Quantifying Memorization Across Neural Language Modelscited by 755
- 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
27 more
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
- 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
- 2022Exploring Length Generalization in Large Language Modelscited by 204
- 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
- 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approachcited by 79
- 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
- 2022Do Language Models Plagiarize?cited by 72
- 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
- 2018ViZDoom Competitions: Playing Doom from Pixelscited by 50
- 2022Things not Written in Text: Exploring Spatial Commonsense from Visual Signalscited by 41
- 2023Multimodal Procedural Planning via Dual Text-Image Promptingcited by 40
- 2023Understanding the Capabilities of Large Language Models for Automated Planningcited by 34
- 2020Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervisioncited by 27
- 2022LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexiacited by 27
- 2024PokeLLMon: A Human-Parity Agent for Pokemon Battles with Large Language Modelscited by 25
- 2023An Evaluation on Large Language Model Outputs: Discourse and Memorizationcited by 20
- 2018Factorized Q-Learning for Large-Scale Multi-Agent Systemscited by 19
- 2022Imagination-Augmented Natural Language Understandingcited by 19
- 2022Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imaginationcited by 18
- 2020Benchmarking End-to-End Behavioural Cloning on Video Gamescited by 15
- 2022Multimedia Generative Script Learning for Task Planningcited by 12
- 2023Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Gamescited by 10
- 2017Autoencoder-augmented Neuroevolution for Visual Doom Playingcited by 8
- 2023"I'd Like to Have an Argument, Please": Argumentative Reasoning in Large Language Modelscited by 4
Built on by
Citing papers per year: 2024: 4; 2025: 2.
6 papers.
- 2024A Survey on Large Language Model-Based Game Agentscited by 106
- 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
- 2024GPT for Games: An Updated Scoping Review (2020-2024)cited by 10
- 2024GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Mapscited by 8
- 2025EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Abilitycited by 6
- 2025GamerAstra: Supporting 2D Non-Twitch Video Games for Blind and Low-Vision Players through a Multi-Agent Frameworkcited by 0
- 2025GamerAstra: Supporting 2D Non-Twitch Video Games for Blind and Low-Vision Players through a Multi-Agent Framework28 Jun 2025
- 2025EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability13 Mar 2025
- 2024GPT for Games: An Updated Scoping Review (2020-2024)1 Nov 2024
- 2024GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Maps10 Oct 2024
- 2024A Survey on Large Language Model-Based Game Agents2 Apr 2024
- 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models1 Apr 2024
Similar, not cited
Close in wording, no citation either way
- 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4
- 2025Vox Deorum: A Hybrid LLM Architecture for 4X / Grand Strategy Game AI -- Lessons from Civilization V
- 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents
- 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information
- 2024Aligning Agents like Large Language Models
- 2024Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Games
- 2025GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games
- 2022Using cognitive psychology to understand GPT-3