# GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environments

arXiv: https://arxiv.org/abs/2602.09007 · HTML: https://mlsanity.com/paper/2602.09007/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 61 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 30 of 46)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents](https://mlsanity.com/paper/2401.10935/) (2024)
- [Leveraging Procedural Generation to Benchmark Reinforcement Learning](https://mlsanity.com/paper/1912.01588/) (2019)
- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [Step1X-Edit: A Practical Framework for General Image Editing](https://mlsanity.com/paper/2504.17761/) (2025)
- [Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis](https://mlsanity.com/paper/2412.04431/) (2024)
- [Seedream 4.0: Toward Next-generation Multimodal Image Generation](https://mlsanity.com/paper/2509.20427/) (2025)
- [Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale](https://mlsanity.com/paper/2409.08264/) (2024)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation](https://mlsanity.com/paper/2407.14505/) (2024)
- [Large Language Model-Brained GUI Agents: A Survey](https://mlsanity.com/paper/2411.18279/) (2024)
- [Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens](https://mlsanity.com/paper/2410.13863/) (2024)
- [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023)
- [LongCat-Image Technical Report](https://mlsanity.com/paper/2512.07584/) (2025)
- [Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing](https://mlsanity.com/paper/2504.02826/) (2025)
- [Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback](https://mlsanity.com/paper/2510.16888/) (2025)
- [NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale](https://mlsanity.com/paper/2508.10711/) (2025)
- [UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokens](https://mlsanity.com/paper/2505.14671/) (2025)
- [Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videos](https://mlsanity.com/paper/2506.05302/) (2025)
- [Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMs](https://mlsanity.com/paper/2510.24514/) (2025)

## Built on by (most cited first, 5 of 5)

- [WebVR: Benchmarking Multimodal LLMs for WebPage Recreation from Videos via Human-Aligned Visual Rubrics](https://mlsanity.com/paper/2603.13391/) (2026)
- [VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System](https://mlsanity.com/paper/2607.27380/) (2026)
- [CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation](https://mlsanity.com/paper/2603.08652/) (2026)
- [GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments](https://mlsanity.com/paper/2609.00048/) (2026)
- [SketchVLM: Vision language models can annotate images to explain thoughts and guide users](https://mlsanity.com/paper/2604.22875/) (2026)

## Abstract (arXiv)

An image generation model used as a graphical user interface (GUI) environment must produce not only a plausible screen but also the correct response to an action. Existing visual-generation benchmarks do not directly assess this combination of functional correctness and temporal consistency in GUIs. We introduce GUI-GenBench, a benchmark of 700 curated samples across five task suites: single-step transitions, multi-step planning, fictional-app generation, real-app trajectories, and coordinate-based grounding. We also introduce GUI-Score, a VLM-based metric that assesses Goal Achievement, Interaction Logic, Content Consistency, UI Plausibility, and Visual Quality. Evaluating 12 image generation models on Chinese and English subsets, we find that strong single-step performance does not consistently extend to coherent multi-step trajectories or accurate grounding. The highest aggregate GUI-Scores are 69.62 on the Chinese subset and 63.16 on the English subset. Qualitative analysis further identifies failures in text rendering, icon interpretation, and coordinate-conditioned transitions. Together, these results distinguish visual plausibility from functional reliability and identify limitations that must be addressed before generated GUIs can serve as dependable interaction environments.