# SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation Evaluation

arXiv: https://arxiv.org/abs/2608.22390 · HTML: https://mlsanity.com/paper/2608.22390/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering](https://mlsanity.com/paper/2403.03163/) (2024)
- [LayoutVAE: Stochastic Scene Layout Generation From a Label Set](https://mlsanity.com/paper/1907.10719/) (2019)
- [pix2code: Generating Code from a Graphical User Interface Screenshot](https://mlsanity.com/paper/1705.07962/) (2017)
- [META-GUI: Towards Multi-modal Conversational Agents on Mobile GUI](https://mlsanity.com/paper/2205.11029/) (2022)
- [ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents](https://mlsanity.com/paper/2507.22827/) (2025)
- [UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs](https://mlsanity.com/paper/2505.09904/) (2025)
- [UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset](https://mlsanity.com/paper/2407.08850/) (2024)
- [Sketch2code: Generating a website from a paper mockup](https://mlsanity.com/paper/1905.13750/) (2019)
- [UI Layout Generation with LLMs Guided by UI Grammar](https://mlsanity.com/paper/2310.15455/) (2023)
- [StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Text](https://mlsanity.com/paper/2406.10621/) (2024)
- [Widget2Code: From Visual Widgets to UI Code via Multimodal LLMs](https://mlsanity.com/paper/2512.19918/) (2025)
- [Generative UI: LLMs are Effective UI Generators](https://mlsanity.com/paper/2604.09577/) (2026)
- [WAFFLE: Finetuning Multi-Modal Models for Automated Front-End Development](https://mlsanity.com/paper/2410.18362/) (2024)
- [GUIDE: LLM-Driven GUI Generation Decomposition for Automated Prototyping](https://mlsanity.com/paper/2502.21068/) (2025)
- [AlignUI: A Method for Designing LLM-Generated UIs Aligned with User Preferences](https://mlsanity.com/paper/2601.17614/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated strong potential in graphical user interface (GUI) generation, but reliable evaluation remains challenging due to uncontrolled data distributions, noisy annotations, and limited layout scenario coverage. To address this, we propose SchemaGUI, a template-based benchmark for controllable GUI generation evaluation. By synthesizing paired natural language instructions and deterministic function-call references from parameterized interface schemas, SchemaGUI can generate thousands of deterministically annotated tasks in seconds without human labeling. Based on 1,000 evaluated instances per scenario and language across six representative bilingual scenarios, we benchmark five mainstream models, including the Qwen3.5 family, Qwen3-Coder-30B, and DeepSeek-R1. Our extensive analysis reveals three key insights. First, precise geometric spatial control remains an important bottleneck; while scaling Qwen3.5 from 4B to 27B improves Schema Feasibility from 91.56% to 99.63%, the Geometry score improves more modestly (from 67.05% to 75.30%). Second, generation difficulty is highly sensitive to layout complexity, with current LLMs excelling at simple sequential arrangements but suffering severe coordinate drift in dense grids and multi-region compositions. Third, thinking mode increases token consumption while generally reducing GUI Score, particularly for smaller models.