mlsanityData to 7 Oct 2026

MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

arXiv 2608.03474 ↗First posted 4 Aug 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

also in 1 more
Abstract (arXiv)

Recent advances in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in web UI generation. However, existing benchmarks predominantly focus on single-turn full-page generation from scratch, overlooking the iterative workflow of real-world frontend engineering, where developers repeatedly reconstruct missing regions and modify localized elements within existing codebases. To bridge this gap, we introduce MT-Web2Code, the first multimodal coding benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification, which contains 102 tasks spanning 16 vertical domains. To construct deterministic repair trajectories without costly turn-level human annotation, we develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages. We further propose a dual-axis evaluation protocol that measures target-region fidelity and the preservation of unaffected content, where regional reconstruction is assessed by a 5-dimensional VLM-based rubric and localized modification by deterministic pixel-grounded alignment. Experiments on 13 frontier coding agents reveal that current agents struggle to faithfully reconstruct target regions while preserving unaffected content, lack fine-grained visual-code alignment for localized edits, and suffer from error snowballing over multiple turns. Beyond benchmarking, our deterministic evaluation metrics provide fine-grained feedback signals that may facilitate future research on training iterative UI coding agents. Our evaluation code and data will soon be released.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 21 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

15 of 21 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026DiffSpot: Can VLMs Spot Fine-Grained Visual Differences in Web Interfaces?28 May
  2. 2025Advancing vision-language models in front-end development via data synthesis3 Mar
  3. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs28 Jun
  4. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering5 Mar

Other references · 11

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2025Qwen3-VL Technical Reportcited by 2,534
  3. 2024A Survey on LLM-as-a-Judgecited by 1,384
  4. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  5. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
  6. 2025A Survey on Code Generation with LLM-based Agentscited by 104
  7. 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 66
  8. 2026Seed2.0 Model Card: Towards Intelligence Frontier for Real-World Complexitycited by 48
  9. 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 43
  10. 2026GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agentscited by 30
  11. 2026Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verificationcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models
  2. 2026Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation
  3. 2026PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides
  4. 2025WebRenderBench: Enhancing Web Interface Generation through Layout-Style Consistency and Reinforcement Learning
  5. 2026Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes
  6. 2025IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?
  7. 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation
  8. 2026LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings