# Towards Human-AI Synergy in UI Design: Supporting Iterative Generation with LLMs

arXiv: https://arxiv.org/abs/2412.20071 · HTML: https://mlsanity.com/paper/2412.20071/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2024.

## Built on by (most cited first, 5 of 5)

- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence](https://mlsanity.com/paper/2606.15932/) (2026)
- [GraphicBench: A Planning Benchmark for Graphic Design with Language Agents](https://mlsanity.com/paper/2504.11571/) (2025)
- [Humans are Missing from AI Coding Agent Research](https://mlsanity.com/paper/2608.12355/) (2026)
- [A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standards](https://mlsanity.com/paper/2505.13766/) (2025)

## Abstract (arXiv)

In automated UI design generation, a key challenge is the lack of support for iterative processes, as most systems focus solely on end-to-end output. This stems from limited capabilities in interpreting design intent and a lack of transparency for refining intermediate results. To better understand these challenges, we conducted a formative study that identified concrete and actionable requirements for supporting iterative design with Generative Tools. Guided by these findings, we propose PrototypeFlow, a human-centered system for automated UI generation that leverages multi-modal inputs and models. PrototypeFlow takes natural language descriptions and layout preferences as input to generate the high-fidelity UI design. At its core is a theme design module that clarifies implicit design intent through prompt enhancement and orchestrates sub-modules for component-level generation. Designers retain full control over inputs, intermediate results, and final prototypes, enabling flexible and targeted refinement by steering generation and directly editing outputs. Our experiments and user studies confirmed the effectiveness and usefulness of our proposed PrototypeFlow.