Deriving and Evaluating a Detailed Taxonomy of Game Bugs

Game development has become an extremely competitive multi-billion-dollar industry. Many games fail even after years of development efforts because of game-breaking bugs that disrupt the game-play and ruin the player experience. The goal of this work is to provide a bug taxonomy for games that will help game developers in developing bug-resistant games, game testers in designing and executing fault-finding test cases, and researchers in evaluating game testing approaches. For this purpose, we performed a Multivocal Literature Review (MLR) by analyzing 436 sources, out of which 189 (78 academic and 111 grey) sources reporting bugs encountered in the game development industry were selected for analysis. We validate the proposed taxonomy by conducting a survey involving different game industry practitioners. The MLR allowed us to finalize a detailed taxonomy of 63 game bug categories in end-user perspective including eight first-tier categories: Gaming Balance, Implementation Response, Network, Sound, Temporal, Unexpected Crash, Navigational, and Non-Temporal faults. We observed that manual approaches towards game testing are still widely used. Only one of the approaches targets sound bugs whereas game balancing and how to incorporate machine learning in game testing is trending in the recent literature. Most of the game testing techniques are specialized and dependent on specific platforms.