mlsanityData to 7 Oct 2026

Will GPT-4 Run DOOM?

arXiv 2403.05468 ↗First posted 8 Mar 2024cs.CL cs.AI cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

We show that GPT-4's reasoning and planning capabilities extend to the 1993 first-person shooter Doom. This large language model (LLM) is able to run and play the game with only a few instructions, plus a textual description--generated by the model itself from screenshots--about the state of the game being observed. We find that GPT-4 can play the game to a passable degree: it is able to manipulate doors, combat enemies, and perform pathing. More complex prompting strategies involving multiple model calls provide better results. While further work is required to enable the LLM to play the game as well as its classical, reinforcement learning-based counterparts, we note that GPT-4 required no training, leaning instead on its own reasoning and observational capabilities. We hope our work pushes the boundaries on intelligent, LLM-based agents in video games. We conclude by discussing the ethical implications of our work.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 39 of its 55 references that are in the map. First posted on arXiv 8 Mar 2024.

Builds on

39 of 55 references.

References · 39

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  8. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  9. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  10. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  11. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  12. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
27 more
  1. 2018On Evaluation of Embodied Navigation Agentscited by 526
  2. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  3. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  4. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  5. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  6. 2022Exploring Length Generalization in Large Language Modelscited by 204
  7. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  8. 2023Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approachcited by 79
  9. 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arenacited by 78
  10. 2022Do Language Models Plagiarize?cited by 72
  11. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  12. 2018ViZDoom Competitions: Playing Doom from Pixelscited by 50
  13. 2022Things not Written in Text: Exploring Spatial Commonsense from Visual Signalscited by 41
  14. 2023Multimodal Procedural Planning via Dual Text-Image Promptingcited by 40
  15. 2023Understanding the Capabilities of Large Language Models for Automated Planningcited by 34
  16. 2020Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervisioncited by 27
  17. 2022LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexiacited by 27
  18. 2024PokeLLMon: A Human-Parity Agent for Pokemon Battles with Large Language Modelscited by 25
  19. 2023An Evaluation on Large Language Model Outputs: Discourse and Memorizationcited by 20
  20. 2018Factorized Q-Learning for Large-Scale Multi-Agent Systemscited by 19
  21. 2022Imagination-Augmented Natural Language Understandingcited by 19
  22. 2022Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imaginationcited by 18
  23. 2020Benchmarking End-to-End Behavioural Cloning on Video Gamescited by 15
  24. 2022Multimedia Generative Script Learning for Task Planningcited by 12
  25. 2023Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Gamescited by 10
  26. 2017Autoencoder-augmented Neuroevolution for Visual Doom Playingcited by 8
  27. 2023"I'd Like to Have an Argument, Please": Argumentative Reasoning in Large Language Modelscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 2.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4
  2. 2025Vox Deorum: A Hybrid LLM Architecture for 4X / Grand Strategy Game AI -- Lessons from Civilization V
  3. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents
  4. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information
  5. 2024Aligning Agents like Large Language Models
  6. 2024Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Games
  7. 2025GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games
  8. 2022Using cognitive psychology to understand GPT-3