IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?
Not on the map. Why?
Abstract (arXiv)
The webpage-to-code task requires models to understand visual representations of webpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic interactions fundamental to real-world web applications. To address this limitation, this paper introduces IWR-Bench, a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video. IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions and featuring diverse interaction complexities (e.g., web games), visual styles, and domains. Aligning with standard web development practices, each task includes not only user interaction videos but also all crawled static assets (e.g., images, videos). This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code. An agent-as-a-judge framework with a comprehensive metric system automatically assesses the functional correctness and visual fidelity of generated webpages. Extensive experiments on 28 LVLMs reveal a significant challenge: the best model achieves an overall score of only 36.35%, as functional correctness (24.39% IFS) lags significantly behind visual fidelity (64.25% VFS). These results highlight critical limitations in current models' ability to reason about temporal dynamics and synthesize event-driven logic, establishing IWR-Bench as a challenging frontier for vision-language research. The benchmark and evaluation code will be made publicly available at https://github.com/SIGMME/IWR-Bench.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 53 of its 59 references that are in the map. First posted on arXiv 29 Sep 2025.
Builds on
53 of 59 references.
References · 53
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
38 more
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
- 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
- 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
- 2025Seed1.5-VL Technical Reportcited by 373
- 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
- 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 339
- 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
- 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
- 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
- 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 203
- 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
- 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
- 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 160
- 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
- 2025PaddleOCR 3.0 Technical Reportcited by 143
- 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
- 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 114
- 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
- 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 100
- 2023SEED-Bench-2: Benchmarking Multimodal Large Language Modelscited by 97
- 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 95
- 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
- 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 72
- 2025VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledgecited by 55
- 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 51
- 2023AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answeringcited by 40
- 2025Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworkscited by 40
- 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agentscited by 26
- 2023A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understandingcited by 18
- 2024IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Webcited by 16
- 2025WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generationcited by 16
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesiscited by 9
- 2026WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Modelscited by 6
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
- 2026Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generationcited by 3
- 2026I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applicationscited by 3
- 2026WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifactscited by 2
- 2026OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documentscited by 2
- 2026Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Gamescited by 0
- 2026UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generationcited by 0
- 2026Framework and Benchmark for Code-Driven Agentic Testing in Web Developmentcited by 0
- 2026Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Developmentcited by 0
- 2026GameReplica: A Benchmark for Black-Box Visual Game Replication by Vision-Language Agentscited by 0
- 2026GameReplica: A Benchmark for Black-Box Visual Game Replication by Vision-Language Agents15 Sep 2026
- 2026Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development2 Sep 2026
- 2026Framework and Benchmark for Code-Driven Agentic Testing in Web Development31 Aug 2026
- 2026OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documents10 Jul 2026
- 2026UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation7 Jul 2026
- 2026Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games17 Jun 2026
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence14 Jun 2026
- 2026WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts2 Jun 2026
- 2026WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis1 Jun 2026
- 2026I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications30 May 2026
- 2026Cookie-Bench: Continuous On-screen Key Interaction Evaluation for Web Generation28 May 2026
- 2026WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models20 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
- 2026LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings
- 2026InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
- 2026Video2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisit
- 2026Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation
- 2026MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification
- 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation
- 2025FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow