mlsanityData to 14 Oct 2026

WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation

arXiv 2508.16763 ↗First posted 22 Aug 2025cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

We present WebMMU, a multilingual benchmark that evaluates three core web tasks: (1) website visual question answering, (2) code editing involving HTML/CSS/JavaScript, and (3) mockup-to-code generation. Unlike prior benchmarks that treat these tasks separately, WebMMU unifies them using expert-annotated, real-world web data to assess models' abilities in complex multi-step reasoning, precise element grounding, and functional UI comprehension and coding. Our evaluation shows that while multimodal large language models (MLLMs) perform well on basic information extraction, they struggle with reasoning and grounding, editing code to preserve functionality, and generating design-to-code that maintains hierarchy and supports multilingual content. These findings reveal key limitations in current MLLMs and underscore the need for improved multimodal and cross-lingual reasoning to build future web agents capable of automating diverse web development tasks.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 42 of its 59 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

42 of 59 references.

References · 42

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  10. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  11. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  12. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
30 more
  1. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  2. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  3. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  4. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 563
  5. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  6. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
  7. 2024WebLINX: Real-World Website Navigation with Multi-Turn Dialoguecited by 180
  8. 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 175
  9. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 158
  10. 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
  11. 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 114
  12. 2021WebQA: Multihop and Multimodal QAcited by 109
  13. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 102
  14. 2021WebSRC: A Dataset for Web-Based Structural Reading Comprehensioncited by 100
  15. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?cited by 100
  16. 2024ScreenAI: A Vision-Language Model for UI and Infographics Understandingcited by 98
  17. 2021UIBert: Learning Generic Multimodal Representations for UI Understandingcited by 95
  18. 2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 76
  19. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
  20. 2024MMInA: Benchmarking Multihop Multimodal Internet Agentscited by 55
  21. 2023Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructionscited by 48
  22. 2021Screen Parsing: Towards Reverse Engineering of UI Models from Screenshotscited by 32
  23. 2024HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Groundingcited by 31
  24. 2023InstructCoder: Instruction Tuning Large Language Models for Code Editingcited by 29
  25. 2024Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototypingcited by 29
  26. 2024Dual-View Visual Contextualization for Web Navigationcited by 24
  27. 2019Sketch2Code: Transformation of Sketches to UI in Real-time Using Deep Neural Networkcited by 23
  28. 2024CodeEditorBench: Evaluating Code Editing Capability of Large Language Modelscited by 23
  29. 2024WebQuest: A Benchmark for Multimodal QA on Web Page Sequencescited by 12
  30. 2022Sketch2FullStack: Generating Skeleton Code of Full Stack Website and Application from Sketch using Deep Learning and Computer Visioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 7.

16 papers.

Sort:

By area:

  1. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  2. 2025IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?cited by 12
  3. 2025Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestrationcited by 9
  4. 2026AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoningcited by 7
  5. 2026WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Modelscited by 6
  6. 2025MPR-GUI: Benchmarking and Enhancing Multilingual Perception and Reasoning in GUI Agentscited by 5
  7. 2025InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generationcited by 4
  8. 2026Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinementcited by 4
  9. 2025Composition-Grounded Data Synthesis for Visual Reasoningcited by 3
  10. 2025WebGen-V Bench: Structured Representation for Enhancing Visual Design in LLM-based Web Generation and Evaluationcited by 3
  11. 2025Can Agent Conquer Web? Exploring the Frontiers of ChatGPT Atlas Agent in Web Gamescited by 1
  12. 2025PyFi: Toward Pyramid-like Financial Image Understanding for VLMs via Adversarial Agentscited by 0
4 more
  1. 2026SEATauBench: Progressively Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languagescited by 0
  2. 2026Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generationcited by 0
  3. 2026Framework and Benchmark for Code-Driven Agentic Testing in Web Developmentcited by 0
  4. 2026AUV-Bench: Aesthetic Understanding and Generation Evaluation for User Interfacescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code
  2. 2026Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
  3. 2025Benchmarking MLLM-based Web Understanding: Reasoning, Robustness and Safety
  4. 2024Harnessing Webpage UIs for Text-Rich Visual Understanding
  5. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping
  6. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering
  7. 2026InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
  8. 2026MARCA: A Checklist-Based Benchmark for Multilingual Web Search