InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
With the advancement of multimodal large language models (MLLMs) and coding agents, the website development has shifted from manual programming to agent-based project-level code synthesis. Existing benchmarks rely on idealized assumptions, especially for well-structured, information-rich inputs and static execution settings. In contrast, real-world development is constrained by a critical bottleneck: the semantic misalignment between ambiguous, low-quality instructions from non-expert users and model understanding, which results in a failure mode that we term blind execution. To address this gap, we introduce InteractWeb-Bench, the first multimodal interactive benchmark for website generation under non-expert low-code user conditions. InteractWeb-Bench introduces four types of user agents and persona-driven instruction perturbations to systematically simulate diverse user behaviors, including ambiguity, redundancy, and contradiction, grounded in requirement engineering defect taxonomies. We develop an interactive execution environment for agents, featuring a unified action space comprising Clarify, Implement, Verify, and Submit, enabling iterative intent refinement, code synthesis, and visual feedback-based validation. Extensive experiments and analysis reveal that frontier MLLM-based agents remain trapped in blind execution, exposing limitations in intent recognition and adaptive interaction.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 21 of its 31 references that are in the map. First posted on arXiv 30 Apr 2026.
Builds on
21 of 31 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
- 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
- 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
- 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
8 more
- 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistancecited by 87
- 2020Metamorphic Testing: A New Approach for Generating Next Test Casescited by 75
- 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
- 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 72
- 2024HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agentcited by 24
- 2026Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User Historycited by 14
- 2025ClarifyMT-Bench: Benchmarking and Improving Multi-Turn Clarification for Conversational Large Language Modelscited by 13
- 2025Can Code Language Models Learn Clarification-Seeking Behaviors?cited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development
- 2026Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
- 2026WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models
- 2026CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception
- 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping
- 2026Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification
- 2025WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning
- 2024AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations