A Survey of Video Game Testing
Not on the map. Why?
Abstract (arXiv)
Video-game projects are notorious for having day-one bugs, no matter how big their budget or team size. The quality of a game is essential for its success. This quality could be assessed and ensured through testing. However, to the best of our knowledge, little is known about video-game testing. In this paper, we want to understand how game developers perform game testing. We investigate, through a survey, the academic and gray literature to identify and report on existing testing processes and how they could automate them. We found that game developers rely, almost exclusively, upon manual play-testing and the testers' intrinsic knowledge. We conclude that current testing processes fall short because of their lack of automation, which seems to be the natural next step to improve the quality of games while maintaining costs. However, the current game-testing techniques may not generalize to different types of games.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2021.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2022: 5; 2023: 2; 2024: 4; 2025: 4; 2026: 3.
18 papers.
Sort:
By area:
- 2022CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learningcited by 15
- 2022Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectorscited by 14
- 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
- 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injectioncited by 9
- 2022Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigationcited by 7
- 2022Learning to Identify Perceptual Bugs in 3D Video Gamescited by 5
- 2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agentscited by 5
- 2024Robust Imitation Learning for Automated Game Testingcited by 4
- 2025Agents in the Sandbox: End-to-End Crash Bug Reproduction for Minecraftcited by 4
- 2025Leveraging LLM Agents for Automated Video Game Testingcited by 4
- 2022Go-Explore Complex 3D Game Environments for Automated Reachability Testingcited by 3
- 2023Preference-conditioned Pixel-based AI Agent For Game Testingcited by 3
6 more
- 2024Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessmentcited by 3
- 2025Human-AI Collaborative Game Testing with Vision Language Modelscited by 1
- 2025Knowledge Graph-enhanced Large Language Model for Incremental Game PlayTestingcited by 1
- 2024Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Gamescited by 0
- 2026CA2: Code-Aware Agent for Automated Game Testingcited by 0
- 2026TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videoscited by 0
- 2026TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videos20 May 2026
- 2026CA2: Code-Aware Agent for Automated Game Testing13 May 2026
- 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injection8 May 2026
- 2025Knowledge Graph-enhanced Large Language Model for Incremental Game PlayTesting4 Nov 2025
- 2025Leveraging LLM Agents for Automated Video Game Testing26 Sep 2025
- 2025Agents in the Sandbox: End-to-End Crash Bug Reproduction for Minecraft25 Mar 2025
- 2025Human-AI Collaborative Game Testing with Vision Language Models20 Jan 2025
- 2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agents1 Oct 2024
- 2024Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessment15 Aug 2024
- 2024Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games6 Jul 2024
- 2024Robust Imitation Learning for Automated Game Testing9 Jan 2024
- 2023Preference-conditioned Pixel-based AI Agent For Game Testing18 Aug 2023
6 more
- 2023Games for Artificial Intelligence Research: A Review and Perspectives26 Apr 2023
- 2022Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors5 Oct 2022
- 2022Go-Explore Complex 3D Game Environments for Automated Reachability Testing1 Sep 2022
- 2022Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation18 Jul 2022
- 2022CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning21 Mar 2022
- 2022Learning to Identify Perceptual Bugs in 3D Video Games25 Feb 2022
- 2024Robust Imitation Learning for Automated Game Testingcited by 4
- 2024Robust Imitation Learning for Automated Game Testing9 Jan 2024
- 2022CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learningcited by 15
- 2022Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectorscited by 14
- 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
- 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injectioncited by 9
- 2022Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigationcited by 7
- 2022Learning to Identify Perceptual Bugs in 3D Video Gamescited by 5
- 2022Go-Explore Complex 3D Game Environments for Automated Reachability Testingcited by 3
- 2023Preference-conditioned Pixel-based AI Agent For Game Testingcited by 3
- 2024Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessmentcited by 3
- 2025Human-AI Collaborative Game Testing with Vision Language Modelscited by 1
- 2025Knowledge Graph-enhanced Large Language Model for Incremental Game PlayTestingcited by 1
- 2024Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Gamescited by 0
12 most cited of 13 All 13 (JSON)
- 2026CA2: Code-Aware Agent for Automated Game Testing13 May 2026
- 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injection8 May 2026
- 2025Knowledge Graph-enhanced Large Language Model for Incremental Game PlayTesting4 Nov 2025
- 2025Human-AI Collaborative Game Testing with Vision Language Models20 Jan 2025
- 2024Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessment15 Aug 2024
- 2024Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games6 Jul 2024
- 2023Preference-conditioned Pixel-based AI Agent For Game Testing18 Aug 2023
- 2023Games for Artificial Intelligence Research: A Review and Perspectives26 Apr 2023
- 2022Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors5 Oct 2022
- 2022Go-Explore Complex 3D Game Environments for Automated Reachability Testing1 Sep 2022
- 2022Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation18 Jul 2022
- 2022CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning21 Mar 2022
12 newest of 13 All 13 (JSON)