Enabling Creative Exploration for Vibe Design Agents
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Vibe design agents translate natural-language requests into interfaces, yet repeated generations often converge on similar visual directions. We present an inference architecture that makes design exploration an explicit stage before implementation. A pre-pass proposes structured design specifications with typicality weights, an external policy samples a direction, and the generator implements it under fixed decoding settings. Candidate elicitation draws on Verbalized Sampling. Applied to UI themes and visual-asset prompts, this architecture provides direct control over aesthetic exploration without changing code-generation temperature. Across 168 benchmark prompts, theme sampling expands observed selection coverage from one option to nearly three and increases rendered visual variation. For visual-asset prompts, the strongest observed LLM-judge win/loss ratio is 1.32, with only a small change in whole-screen similarity. Localized aesthetic choices can thus influence judged preference even when the overall screen remains similar. We further evaluate joint theme and asset sampling in a randomized online study conducted over nearly two months, covering 3.39 million tasks and 87,416 labeled conversations. These studies show how explicit design specifications support exploration of alternative UI concepts and help us evaluate their effects on rendered interfaces, judged preference, and user interaction.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 44 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
32 of 44 references.
References · 32
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
- 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
- 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
- 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
- 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 114
- 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 105
- 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
- 2024On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularizationcited by 72
- 2025Diverse Preference Optimizationcited by 52
20 more
- 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 51
- 2025DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generationcited by 45
- 2023Prompting is not a substitute for probability measurements in large language modelscited by 44
- 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 44
- 2021Screen Parsing: Towards Reverse Engineering of UI Models from Screenshotscited by 32
- 2025FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflowcited by 28
- 2024UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedbackcited by 27
- 2024MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designscited by 24
- 2025UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designscited by 22
- 2025FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluationcited by 20
- 2024Calibrating Verbalized Probabilities for Large Language Modelscited by 10
- 2024Misty: UI Prototyping Through Interactive Conceptual Blendingcited by 9
- 2025Asking a Language Model for Diverse Responsescited by 9
- 2024MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototypingcited by 6
- 2024DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Modelscited by 6
- 2025EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generationcited by 5
- 2026ComUICoder: Component-based Reusable UI Code Generation for Complex Websites via Semantic Segmentation and Element-wise Feedbackcited by 5
- 2024Frontend Diffusion: Exploring Intent-Based User Interfaces through Abstract-to-Detailed Task Transitionscited by 3
- 2026The Human Creativity Benchmarkcited by 2
- 2021Spacewalker: Rapid UI Design Exploration Using Lightweight Markup Enhancement and Crowd Genetic Programmingcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback
- 2025SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AI
- 2026UI Remix: Supporting UI Design Through Interactive Example Retrieval and Remixing ↗
- 2025Portal UX Agent -- A Plug-and-Play Engine for Rendering UIs from Natural Language Specifications
- 2025WebVIA: A Web-based Vision-Language Agentic Framework for Interactive and Verifiable UI-to-Code Generation
- 2026A Visionary Look at Vibe Researching
- 2024Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping
- 2026Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design