# UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs

arXiv: https://arxiv.org/abs/2505.09904 · HTML: https://mlsanity.com/paper/2505.09904/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2025.

## Built on by (most cited first, 22 of 22)

- [ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents](https://mlsanity.com/paper/2507.22827/) (2025)
- [P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark](https://mlsanity.com/paper/2505.17104/) (2025)
- [Widget2Code: From Visual Widgets to UI Code via Multimodal LLMs](https://mlsanity.com/paper/2512.19918/) (2025)
- [VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generation](https://mlsanity.com/paper/2602.05998/) (2026)
- [Generative UI: LLMs are Effective UI Generators](https://mlsanity.com/paper/2604.09577/) (2026)
- [EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation](https://mlsanity.com/paper/2509.12159/) (2025)
- [Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation](https://mlsanity.com/paper/2608.24138/) (2026)
- [Paper2Web: Let's Make Your Paper Alive!](https://mlsanity.com/paper/2510.15842/) (2025)
- [TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework](https://mlsanity.com/paper/2510.17163/) (2025)
- [1D-Bench: A Benchmark for Iterative UI Code Generation with Visual Feedback in Real-World](https://mlsanity.com/paper/2602.18548/) (2026)
- [Video2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisit](https://mlsanity.com/paper/2606.20711/) (2026)
- [From Visual Widgets to UI Code: Efficient Tool-Grounded Generation](https://mlsanity.com/paper/2608.12611/) (2026)
- [Reverse Browser: Vector-Image-to-Code Generator](https://mlsanity.com/paper/2509.05394/) (2025)
- [DOne: Decoupling Structure and Rendering for High-Fidelity Design-to-Code Generation](https://mlsanity.com/paper/2604.01226/) (2026)
- [From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generation](https://mlsanity.com/paper/2604.27969/) (2026)
- [ProductWebGen: Benchmarking Multimodal Product Webpage Generation](https://mlsanity.com/paper/2606.01022/) (2026)
- [TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics](https://mlsanity.com/paper/2606.03626/) (2026)
- [Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation](https://mlsanity.com/paper/2608.03691/) (2026)
- [SchemaGUI: A Schema-Driven Benchmark for Controllable GUI Generation Evaluation](https://mlsanity.com/paper/2608.22390/) (2026)
- [Editable Visual Design](https://mlsanity.com/paper/2609.04034/) (2026)
- [Enabling Creative Exploration for Vibe Design Agents](https://mlsanity.com/paper/2609.15078/) (2026)
- [Beyond Static Charts: Can Language and Vision Language Models Generate Interactive Data Visualization Interfaces?](https://mlsanity.com/paper/2609.26208/) (2026)

## Abstract (arXiv)

Automating the synthesis of User Interfaces (UIs) plays a crucial role in enhancing productivity and accelerating the development lifecycle, reducing both development time and manual effort. Recently, the rapid development of Multimodal Large Language Models (MLLMs) has made it possible to generate front-end Hypertext Markup Language (HTML) code directly from webpage designs. However, real-world webpages encompass not only a diverse array of HTML tags but also complex stylesheets, resulting in significantly lengthy code. The lengthy code poses challenges for the performance and efficiency of MLLMs, especially in capturing the structural information of UI designs. To address these challenges, this paper proposes UICopilot, a novel approach to automating UI synthesis via hierarchical code generation from webpage designs. The core idea of UICopilot is to decompose the generation process into two stages: first, generating the coarse-grained HTML hierarchical structure, followed by the generation of fine-grained code. To validate the effectiveness of UICopilot, we conduct experiments on a real-world dataset, i.e., WebCode2M. Experimental results demonstrate that UICopilot significantly outperforms existing baselines in both automatic evaluation metrics and human evaluations. Specifically, statistical analysis reveals that the majority of human annotators prefer the webpages generated by UICopilot over those produced by GPT-4V.