# MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling

arXiv: https://arxiv.org/abs/2405.07090 · HTML: https://mlsanity.com/paper/2405.07090/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2024.

## Built on by (most cited first, 9 of 9)

- [AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents](https://mlsanity.com/paper/2407.17490/) (2024)
- [Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review](https://mlsanity.com/paper/2501.12557/) (2025)
- [AXIS: Efficient Human-Agent-Computer Interaction with API-First LLM-Based Agents](https://mlsanity.com/paper/2409.17140/) (2024)
- [Agent for User: Testing Multi-User Interactive Features in TikTok](https://mlsanity.com/paper/2504.15474/) (2025)
- [DuetUI: A Bidirectional Context Loop for Human-Agent Co-Generation of Task-Oriented Interfaces](https://mlsanity.com/paper/2509.13444/) (2025)
- [PixelWeb: The First Web GUI Dataset with Pixel-Wise Labels](https://mlsanity.com/paper/2504.16419/) (2025)
- [Breaking Single-Tester Limits: Multi-Agent LLMs for Multi-User Feature Testing](https://mlsanity.com/paper/2506.17539/) (2025)
- [ControlGUI: Guiding Generative GUI Exploration through Perceptual Visual Flow](https://mlsanity.com/paper/2502.03330/) (2025)
- [UISearch: Graph-Based Embeddings for Multimodal Enterprise UI Screenshots Retrieval](https://mlsanity.com/paper/2511.19380/) (2025)

## Abstract (arXiv)

The importance of computational modeling of mobile user interfaces (UIs) is undeniable. However, these require a high-quality UI dataset. Existing datasets are often outdated, collected years ago, and are frequently noisy with mismatches in their visual representation. This presents challenges in modeling UI understanding in the wild. This paper introduces a novel approach to automatically mine UI data from Android apps, leveraging Large Language Models (LLMs) to mimic human-like exploration. To ensure dataset quality, we employ the best practices in UI noise filtering and incorporate human annotation as a final validation step. Our results demonstrate the effectiveness of LLMs-enhanced app exploration in mining more meaningful UIs, resulting in a large dataset MUD of 18k human-annotated UIs from 3.3k apps. We highlight the usefulness of MUD in two common UI modeling tasks: element detection and UI retrieval, showcasing its potential to establish a foundation for future research into high-quality, modern UIs.