# GUISpector: An MLLM Agent Framework for Automated Verification of Natural Language Requirements in GUI Prototypes

arXiv: https://arxiv.org/abs/2510.04791 · HTML: https://mlsanity.com/paper/2510.04791/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 7 of 7)

- [PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback](https://mlsanity.com/paper/2412.03578/) (2024)
- [Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing](https://mlsanity.com/paper/2305.09434/) (2023)
- [Enabling Cost-Effective UI Automation Testing with Retrieval-Based LLMs: A Case Study in WeChat](https://mlsanity.com/paper/2409.07829/) (2024)
- [GIFdroid: Automated Replay of Visual Bug Reports for Android Apps](https://mlsanity.com/paper/2112.04128/) (2021)
- [Interlinking User Stories and GUI Prototyping: A Semi-Automatic LLM-based Approach](https://mlsanity.com/paper/2406.08120/) (2024)
- [Zero-Shot Prompting Approaches for LLM-based Graphical User Interface Generation](https://mlsanity.com/paper/2412.11328/) (2024)
- [Agent for User: Testing Multi-User Interactive Features in TikTok](https://mlsanity.com/paper/2504.15474/) (2025)

## Built on by (most cited first, 1 of 1)

- [Can LLMs Fix It Without Code? Toward Automated Verification of No-Code Bug Fixes](https://mlsanity.com/paper/2610.11963/) (2026)

## Abstract (arXiv)

GUIs are foundational to interactive systems and play a pivotal role in early requirements elicitation through prototyping. Ensuring that GUI implementations fulfill NL requirements is essential for robust software engineering, especially as LLM-driven programming agents become increasingly integrated into development workflows. Existing GUI testing approaches, whether traditional or LLM-driven, often fall short in handling the complexity of modern interfaces, and typically lack actionable feedback and effective integration with automated development agents. In this paper, we introduce GUISpector, a novel framework that leverages a multi-modal (M)LLM-based agent for the automated verification of NL requirements in GUI prototypes. First, GUISpector adapts a MLLM agent to interpret and operationalize NL requirements, enabling to autonomously plan and execute verification trajectories across GUI applications. Second, GUISpector systematically extracts detailed NL feedback from the agent's verification process, providing developers with actionable insights that can be used to iteratively refine the GUI artifact or directly inform LLM-based code generation in a closed feedback loop. Third, we present an integrated tool that unifies these capabilities, offering practitioners an accessible interface for supervising verification runs, inspecting agent rationales and managing the end-to-end requirements verification process. We evaluated GUISpector on a comprehensive set of 150 requirements based on 900 acceptance criteria annotations across diverse GUI applications, demonstrating effective detection of requirement satisfaction and violations and highlighting its potential for seamless integration of actionable feedback into automated LLM-driven development workflows. The video presentation of GUISpector is available at: https://youtu.be/JByYF6BNQeE, showcasing its main capabilities.