TaoD2C-Bench: Benchmarking MLLMs for Industrial UI Code Generation Beyond Visual Fidelity
On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection
Abstract (arXiv)
A key challenge for multimodal large language models (MLLMs) is moving beyond visual recognition to constraint-aware cross-modal reasoning. This involves combining visual cues with information from other modalities to understand elements' relationships under domain-specific rules. This challenge is acutely evident in industrial design-to-code (D2C), which converts user interface (UI) designs into code and requires MLLMs to connect design images with disorganized layer metadata, infer component and layout implementation requirements, and realize them in code under target-library constraints. However, these capabilities remain insufficiently evaluated in realistic industrial settings. To fill this gap, we present TaoD2C-Bench, a benchmark for evaluating MLLMs' ability to generate UI code that satisfies implementation requirements in industrial applications. The TaoD2C dataset consists of 2,861 production designs from 17 commercial platforms with 97,652 expert annotations across four categories: Component, Group, Alignment, and Position. These annotations distinguish required constraints from permitted implementation choices. TaoD2C-Bench defines three tasks: end-to-end UI code generation, requirement inference, and requirement realization. Evaluating eight MLLMs reveals substantial gaps in generating UI code that satisfies implementation requirements, alongside distinct performance profiles in inference and realization. We further show that MLLMs' visual reconstruction ability does not necessarily imply an ability to generate code that meets these requirements. We release TaoD2C to support research on industrial UI code generation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 50 references that are in the map. First posted on arXiv 7 Oct 2026.
Builds on
17 of 50 references.
Closest ancestors · 4
Same direction, newest first
- 2026Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation25 Aug
- 2026UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization30 Jun
- 2025Advancing vision-language models in front-end development via data synthesis3 Mar
- 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset14 Mar
Other references · 13
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2025DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generationcited by 41
- 2024UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedbackcited by 26
- 2025FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflowcited by 25
- 2024Bridging Design and Development with Automated Declarative UI Code Generationcited by 18
- 2018Automated Reporting of GUI Design Violations for Mobile Appscited by 15
- 2025UI2Code^N: UI-to-Code Generation as Interactive Visual Optimizationcited by 12
- 2025WebVIA: A Web-based Vision-Language Agentic Framework for Interactive and Verifiable UI-to-Code Generationcited by 8
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 7
- 2026Vision-Guided Iterative Refinement for Frontend Code Generationcited by 3
- 2025WebRenderBench: Enhancing Web Interface Generation through Layout-Style Consistency and Reinforcement Learningcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering
- 2025MLLM-Based UI2Code Automation Guided by UI Layout Information
- 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents
- 2026MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction
- 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping
- 2025CANVAS: A Benchmark for Vision-Language Models on Tool-Based User Interface Design
- 2025Structuring GUI Elements through Vision Language Models: Towards Action Space Generation
- 2026DOne: Decoupling Structure and Rendering for High-Fidelity Design-to-Code Generation