# GLIB: Towards Automated Test Oracle for Graphically-Rich Applications

arXiv: https://arxiv.org/abs/2106.10507 · HTML: https://mlsanity.com/paper/2106.10507/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 19 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Convolutional Networks with Dense Connectivity](https://mlsanity.com/paper/2001.02394/) (2020)
- [Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps](https://mlsanity.com/paper/1802.02312/) (2018)
- [Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?](https://mlsanity.com/paper/2008.05132/) (2020)
- [Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learning](https://mlsanity.com/paper/2003.00380/) (2020)
- [Wireframe-Based UI Design Search Through Image Autoencoder](https://mlsanity.com/paper/2103.07085/) (2021)
- [Owl Eyes: Spotting UI Display Issues via Visual Understanding](https://mlsanity.com/paper/2009.01417/) (2020)
- [StoryDroid: Automated Generation of Storyboard for Android Apps](https://mlsanity.com/paper/1902.00476/) (2019)
- [From Lost to Found: Discover Missing UI Design Semantics through Recovering Missing Tags](https://mlsanity.com/paper/2008.06895/) (2020)

## Built on by (most cited first, 9 of 9)

- [GlitchBench: Can large multimodal models detect video game glitches?](https://mlsanity.com/paper/2312.05291/) (2023)
- [PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos](https://mlsanity.com/paper/2412.01800/) (2024)
- [Less Cybersickness, Please: Demystifying and Detecting Stereoscopic Visual Inconsistencies in Virtual Reality Apps](https://mlsanity.com/paper/2406.09313/) (2024)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [The Integration of Machine Learning into Automated Test Generation: A Systematic Mapping Study](https://mlsanity.com/paper/2206.10210/) (2022)
- [Human-AI Collaborative Game Testing with Vision Language Models](https://mlsanity.com/paper/2501.11782/) (2025)
- [Order from Chaos: Physical World Understanding from Glitchy Gameplay Videos](https://mlsanity.com/paper/2601.16471/) (2026)
- [Runtime Failure Hunting for Physics Engine Based Software Systems: How Far Can We Go?](https://mlsanity.com/paper/2507.22099/) (2025)
- [CA2: Code-Aware Agent for Automated Game Testing](https://mlsanity.com/paper/2605.13918/) (2026)

## Abstract (arXiv)

Graphically-rich applications such as games are ubiquitous with attractive visual effects of Graphical User Interface (GUI) that offers a bridge between software applications and end-users. However, various types of graphical glitches may arise from such GUI complexity and have become one of the main component of software compatibility issues. Our study on bug reports from game development teams in NetEase Inc. indicates that graphical glitches frequently occur during the GUI rendering and severely degrade the quality of graphically-rich applications such as video games. Existing automated testing techniques for such applications focus mainly on generating various GUI test sequences and check whether the test sequences can cause crashes. These techniques require constant human attention to captures non-crashing bugs such as bugs causing graphical glitches. In this paper, we present the first step in automating the test oracle for detecting non-crashing bugs in graphically-rich applications. Specifically, we propose textttGLIB based on a code-based data augmentation technique to detect game GUI glitches. We perform an evaluation of textttGLIB on 20 real-world game apps (with bug reports available) and the result shows that textttGLIB can achieve 100% precision and 99.5% recall in detecting non-crashing bugs such as game GUI glitches. Practical application of textttGLIB on another 14 real-world games (without bug reports) further demonstrates that textttGLIB can effectively uncover GUI glitches, with 48 of 53 bugs reported by textttGLIB having been confirmed and fixed so far.