# Automatically Detecting Visual Bugs in HTML5 <canvas> Games

arXiv: https://arxiv.org/abs/2208.02335 · HTML: https://mlsanity.com/paper/2208.02335/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2022.

## Built on by (most cited first, 5 of 5)

- [Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors](https://mlsanity.com/paper/2210.02506/) (2022)
- [Less Cybersickness, Please: Demystifying and Detecting Stereoscopic Visual Inconsistencies in Virtual Reality Apps](https://mlsanity.com/paper/2406.09313/) (2024)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs](https://mlsanity.com/paper/2406.04482/) (2024)
- [Runtime Failure Hunting for Physics Engine Based Software Systems: How Far Can We Go?](https://mlsanity.com/paper/2507.22099/) (2025)

## Abstract (arXiv)

The HTML5 <canvas> is used to display high quality graphics in web applications such as web games (i.e., <canvas> games). However, automatically testing <canvas> games is not possible with existing web testing techniques and tools, and manual testing is laborious. Many widely used web testing tools rely on the Document Object Model (DOM) to drive web test automation, but the contents of the <canvas> are not represented in the DOM. The main alternative approach, snapshot testing, involves comparing oracle snapshot images with test-time snapshot images using an image similarity metric to catch visual bugs, i.e., bugs in the graphics of the web application. However, creating and maintaining oracle snapshot images for <canvas> games is onerous, defeating the purpose of test automation. In this paper, we present a novel approach to automatically detect visual bugs in <canvas> games. By leveraging an internal representation of objects on the <canvas>, we decompose snapshot images into a set of object images, each of which is compared with a respective oracle asset (e.g., a sprite) using four similarity metrics: percentage overlap, mean squared error, structural similarity, and embedding similarity. We evaluate our approach by injecting 24 visual bugs into a custom <canvas> game, and find that our approach achieves an accuracy of 100%, compared to an accuracy of 44.6% with traditional snapshot testing.