mlsanityData to 7 Oct 2026

GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environments

arXiv 2602.09007 ↗First posted 9 Feb 2026cs.AI cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

An image generation model used as a graphical user interface (GUI) environment must produce not only a plausible screen but also the correct response to an action. Existing visual-generation benchmarks do not directly assess this combination of functional correctness and temporal consistency in GUIs. We introduce GUI-GenBench, a benchmark of 700 curated samples across five task suites: single-step transitions, multi-step planning, fictional-app generation, real-app trajectories, and coordinate-based grounding. We also introduce GUI-Score, a VLM-based metric that assesses Goal Achievement, Interaction Logic, Content Consistency, UI Plausibility, and Visual Quality. Evaluating 12 image generation models on Chinese and English subsets, we find that strong single-step performance does not consistently extend to coherent multi-step trajectories or accurate grounding. The highest aggregate GUI-Scores are 69.62 on the Chinese subset and 63.16 on the English subset. Qualitative analysis further identifies failures in text rendering, icon interpretation, and coordinate-conditioned transitions. Together, these results distinguish visual plausibility from functional reliability and identify limitations that must be addressed before generated GUIs can serve as dependable interaction environments.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 46 of its 61 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

46 of 61 references.

References · 46

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  9. 2025Qwen-Image Technical Reportcited by 973
  10. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  11. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  12. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
34 more
  1. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  2. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
  3. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  4. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  5. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  6. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 212
  7. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  8. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 177
  9. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  10. 2024Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokenscited by 147
  11. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 136
  12. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  13. 2025LongCat-Image Technical Reportcited by 92
  14. 2025Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedbackcited by 78
  15. 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 55
  16. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 55
  17. 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 49
  18. 2025Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMscited by 41
  19. 2025Are Video Models Ready as Zero-Shot Reasoners? An Empirical Study with the MME-CoF Benchmarkcited by 41
  20. 2025GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agentcited by 35
  21. 2025ViMo: A Generative Visual GUI World Model for App Agentscited by 33
  22. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipescited by 22
  23. 2025UI-E2I-Synth: Advancing GUI Grounding with Large-Scale Instruction Synthesiscited by 20
  24. 2025Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skillscited by 18
  25. 2025MobileWorldBench: Towards Semantic World Modeling For Mobile Agentscited by 18
  26. 2021GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial Networkscited by 17
  27. 2025GUI-ReWalk: Massive Data Generation for GUI Agent via Stochastic Exploration and Intent-Aware Reasoningcited by 12
  28. 2026How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editingcited by 12
  29. 2023Boosting GUI Prototyping with Diffusion Modelscited by 10
  30. 2022GANSpiration: Balancing Targeted and Serendipitous Inspiration in User Interface Design with Style-Based Generative Adversarial Networkcited by 6
  31. 2024Zero-Shot Prompting Approaches for LLM-based Graphical User Interface Generationcited by 4
  32. 2025GUIDE: LLM-Driven GUI Generation Decomposition for Automated Prototypingcited by 4
  33. 2025GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learningcited by 4
  34. 2023GUILGET: GUI Layout GEneration with Transformercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.