# Detecting and Summarizing GUI Changes in Evolving Mobile Apps

arXiv: https://arxiv.org/abs/1807.09440 · HTML: https://mlsanity.com/paper/1807.09440/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2018.

## Built on by (most cited first, 4 of 4)

- [An Empirical Investigation into the Use of Image Captioning for Automated Software Documentation](https://mlsanity.com/paper/2301.01224/) (2023)
- [Reimagining Application User Interface (UI) Design using Deep Learning Methods: Challenges and Opportunities](https://mlsanity.com/paper/2303.13055/) (2023)
- [MotorEase: Automated Detection of Motor Impairment Accessibility Issues in Mobile App UIs](https://mlsanity.com/paper/2403.13690/) (2024)
- [AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding](https://mlsanity.com/paper/2404.01240/) (2024)

## Abstract (arXiv)

Mobile applications have become a popular software development domain in recent years due in part to a large user base, capable hardware, and accessible platforms. However, mobile developers also face unique challenges, including pressure for frequent releases to keep pace with rapid platform evolution, hardware iteration, and user feedback. Due to this rapid pace of evolution, developers need automated support for documenting the changes made to their apps in order to aid in program comprehension. One of the more challenging types of changes to document in mobile apps are those made to the graphical user interface (GUI) due to its abstract, pixel-based representation. In this paper, we present a fully automated approach, called GCAT, for detecting and summarizing GUI changes during the evolution of mobile apps. GCAT leverages computer vision techniques and natural language generation to accurately and concisely summarize changes made to the GUI of a mobile app between successive commits or releases. We evaluate the performance of our approach in terms of its precision and recall in detecting GUI changes compared to developer specified changes, and investigate the utility of the generated change reports in a controlled user study. Our results indicate that GCAT is capable of accurately detecting and classifying GUI changes - outperforming developers - while providing useful documentation.