mlsanityData to 14 Oct 2026

WebVIA: A Web-based Vision-Language Agentic Framework for Interactive and Verifiable UI-to-Code Generation

arXiv 2511.06251 ↗First posted 9 Nov 2025cs.SE cs.AIcited by 10

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

User interface (UI) development requires translating design mockups into functional code, a process that remains repetitive and labor-intensive. While recent Vision-Language Models (VLMs) automate UI-to-Code generation, they generate only static HTML/CSS/JavaScript layouts lacking interactivity. To address this, we propose WebVIA, the first agentic framework for interactive UI-to-Code generation and validation. The framework comprises three components: 1) an exploration agent to capture multi-state UI screenshots; 2) a UI2Code model that generates executable interactive code; 3) a validation module that verifies the interactivity. Experiments demonstrate that WebVIA-Agent achieves more stable and accurate UI exploration than general-purpose agents (e.g., Gemini-2.5-Pro). In addition, our fine-tuned WebVIA-UI2Code models exhibit substantial improvements in generating executable and interactive HTML/CSS/JavaScript code, outperforming their base counterparts across both interactive and static UI2Code benchmarks. Our code and models are available at hrefhttps://zheny2751-dotcom.github.io/webvia.github.io/texttthttps://webvia.github.io.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 22 references that are in the map. First posted on arXiv 9 Nov 2025.

Builds on

14 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents30 Jul

Other references · 13

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  3. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  4. 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
  5. 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 114
  6. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 51
  7. 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 44
  8. 2025FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflowcited by 28
  9. 2024Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approachcited by 24
  10. 2024Bridging Design and Development with Automated Declarative UI Code Generationcited by 19
  11. 2025MLLM-Based UI2Code Automation Guided by UI Layout Informationcited by 17
  12. 2025Advancing vision-language models in front-end development via data synthesiscited by 16
1 more
  1. 2024WAFFLE: Finetuning Multi-Modal Models for Automated Front-End Developmentcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping
  2. 2025WebRenderBench: Enhancing Web Interface Generation through Layout-Style Consistency and Reinforcement Learning
  3. 2026VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents
  4. 2026Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development
  5. 2026WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness
  6. 2025Portal UX Agent -- A Plug-and-Play Engine for Rendering UIs from Natural Language Specifications
  7. 2026Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification
  8. 2025FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedback