# DOne: Decoupling Structure and Rendering for High-Fidelity Design-to-Code Generation

arXiv: https://arxiv.org/abs/2604.01226 · HTML: https://mlsanity.com/paper/2604.01226/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 19 of 19)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset](https://mlsanity.com/paper/2403.09029/) (2024)
- [pix2code: Generating Code from a Graphical User Interface Screenshot](https://mlsanity.com/paper/1705.07962/) (2017)
- [Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs](https://mlsanity.com/paper/2406.20098/) (2024)
- [Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps](https://mlsanity.com/paper/1802.02312/) (2018)
- [Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots](https://mlsanity.com/paper/2109.08763/) (2021)
- [Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?](https://mlsanity.com/paper/2008.05132/) (2020)
- [ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents](https://mlsanity.com/paper/2507.22827/) (2025)
- [Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach](https://mlsanity.com/paper/2406.16386/) (2024)
- [MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs](https://mlsanity.com/paper/2412.15310/) (2024)
- [UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs](https://mlsanity.com/paper/2505.09904/) (2025)
- [Using Deep Learning to Generate Complete Log Statements](https://mlsanity.com/paper/2201.04837/) (2022)
- [Automated Reporting of GUI Design Violations for Mobile Apps](https://mlsanity.com/paper/1802.04732/) (2018)
- [Exploring Distributional Shifts in Large Language Models for Code Analysis](https://mlsanity.com/paper/2303.09128/) (2023)

## Abstract (arXiv)

While Vision Language Models (VLMs) have shown promise in Design-to-Code generation, they suffer from a "holistic bottleneck-failing to reconcile high-level structural hierarchy with fine-grained visual details, often resulting in layout distortions or generic placeholders. To bridge this gap, we propose DOne, an end-to-end framework that decouples structure understanding from element rendering. DOne introduces (1) a learned layout segmentation module to decompose complex designs, avoiding the limitations of heuristic cropping; (2) a specialized hybrid element retriever to handle the extreme aspect ratios and densities of UI components; and (3) a schema-guided generation paradigm that bridges layout and code. To rigorously assess performance, we introduce HiFi2Code, a benchmark featuring significantly higher layout complexity than existing datasets. Extensive evaluations on the HiFi2Code demonstrate that DOne outperforms exiting methods in both high-level visual similarity (e.g., over 10% in GPT Score) and fine-grained element alignment. Human evaluations confirm a 3 times productivity gain with higher visual fidelity.