mlsanityData to 14 Oct 2026

IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?

arXiv 2509.24709 ↗First posted 29 Sep 2025cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

The webpage-to-code task requires models to understand visual representations of webpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic interactions fundamental to real-world web applications. To address this limitation, this paper introduces IWR-Bench, a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video. IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions and featuring diverse interaction complexities (e.g., web games), visual styles, and domains. Aligning with standard web development practices, each task includes not only user interaction videos but also all crawled static assets (e.g., images, videos). This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code. An agent-as-a-judge framework with a comprehensive metric system automatically assesses the functional correctness and visual fidelity of generated webpages. Extensive experiments on 28 LVLMs reveal a significant challenge: the best model achieves an overall score of only 36.35%, as functional correctness (24.39% IFS) lags significantly behind visual fidelity (64.25% VFS). These results highlight critical limitations in current models' ability to reason about temporal dynamics and synthesize event-driven logic, establishing IWR-Bench as a challenging frontier for vision-language research. The benchmark and evaluation code will be made publicly available at https://github.com/SIGMME/IWR-Bench.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 53 of its 59 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

53 of 59 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  12. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
38 more
  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  2. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  3. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  4. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  5. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  6. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  7. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  8. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  9. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  10. 2025Seed1.5-VL Technical Reportcited by 373
  11. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  12. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 339
  13. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  14. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
  15. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  16. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 203
  17. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
  18. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
  19. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
  20. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Modelscited by 160
  21. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  22. 2025PaddleOCR 3.0 Technical Reportcited by 143
  23. 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
  24. 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 114
  25. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
  26. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 100
  27. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Modelscited by 97
  28. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 95
  29. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
  30. 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 72
  31. 2025VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledgecited by 55
  32. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 51
  33. 2023AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answeringcited by 40
  34. 2025Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworkscited by 40
  35. 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agentscited by 26
  36. 2023A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understandingcited by 18
  37. 2024IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Webcited by 16
  38. 2025WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generationcited by 16

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 12.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
  2. 2026LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings
  3. 2026InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
  4. 2026Video2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisit
  5. 2026Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation
  6. 2026MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification
  7. 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation
  8. 2025FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow