mlsanityData to 14 Oct 2026

UI2Code^N: UI-to-Code Generation as Interactive Visual Optimization

arXiv 2511.08195 ↗First posted 11 Nov 2025cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

UI-to-code aims to translate UI screenshots into executable front-end code. Despite progress with vision-language models (VLMs), most existing methods formulate UI-to-code as a single-pass generation, which mismatches real-world UI development that is inherently iterative and feedback-driven. We reformulate UI-to-code as an interactive visual optimization problem, where code generation is embedded in a closed-loop process of execution, visual inspection, and iterative refinement driven by rendered visual feedback. To address the non-differentiability of visual objectives and the noise of absolute visual evaluators, we propose Relative Visual Policy Optimization (RVPO), a preference-based reinforcement learning method that optimizes relative visual rankings among rendered candidates under execution feedback. We instantiate this paradigm in UI2Code^N, an open-source 9B model trained via continual pre-training, supervised fine-tuning, and reinforcement learning. Experiments demonstrate state-of-the-art performance on UI drafting, UI polishing, and UI editing benchmarks, even outperforming larger models, with performance consistently improving through iterative visual optimization. Our code and models are available at https://github.com/zai-org/UI2Code_N.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 19 of its 22 references that are in the map. First posted on arXiv 11 Nov 2025.

Builds on

19 of 22 references.

References · 19

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  5. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  6. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  7. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  8. 2025Kimi-VL Technical Reportcited by 388
  9. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  10. 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
  11. 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 114
  12. 2024ScreenAI: A Vision-Language Model for UI and Infographics Understandingcited by 98
7 more
  1. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
  2. 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 72
  3. 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 44
  4. 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agentscited by 26
  5. 2024Bridging Design and Development with Automated Declarative UI Code Generationcited by 19
  6. 2025Advancing vision-language models in front-end development via data synthesiscited by 16
  7. 2025Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferencescited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generation
  2. 2025VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learning
  3. 2026UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
  4. 20261D-Bench: A Benchmark for Iterative UI Code Generation with Visual Feedback in Real-World
  5. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning
  6. 2026FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
  7. 2026VISTA: View-Consistent Self-Verified Training for GUI Grounding
  8. 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward