mlsanityData to 7 Oct 2026

MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction

arXiv 2607.19910 ↗First posted 22 Jul 2026cs.CV cs.HCcited by 0

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Multimodal large language models (MLLMs) are increasingly expected to automate visualization development by generating code directly from visual designs. However, existing evaluations mainly focus on single-chart generation and overlook coordinated multi-view interface construction, which requires joint reasoning about data semantics, view coordination, and interaction logic. Consequently, MLLM capabilities in this setting remain underexplored, and the field lacks a dedicated benchmark for systematic assessment. We introduce MV-Bench, a benchmark for evaluating MLLMs on coordinated multi-view interface construction. Instead of relying on incomplete or inconsistent open-source implementations, we use Tableau workbook files as ground truth because they explicitly encode data bindings, visual mappings, and interactions. We develop a multi-stage pipeline that converts these specifications into executable web interfaces through structured intermediate representations. The benchmark contains 92 base interfaces and 1,048 verified instances created by recombining chart types, datasets, and interaction patterns. Each instance includes executable code, a rendered interface, a dataset, and interaction annotations. We evaluate five state-of-the-art MLLMs in a single-pass setting using metrics for visual fidelity, data binding correctness, and interaction completeness. The strongest model achieves 75.45 percent accuracy in visual layout reproduction, but only 21.71 percent in data binding and 11.68 percent in interaction completeness. These results show that current MLLMs can reproduce visual appearance but remain limited in generating the data semantics and interactive logic required by coordinated multi-view interfaces. Iterative refinement improves code executability but does not substantially reduce the gap in data binding and interaction generation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 53 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

31 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping21 Oct
  2. 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset14 Mar
  3. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering5 Mar
  4. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning19 Feb
  5. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Charts26 Dec
  6. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generation27 Nov

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2024GPT-4o System Cardcited by 4,467
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  5. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  6. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  7. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  8. 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 113
  9. 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualizationcited by 98
  10. 2023LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Modelscited by 80
  11. 2023ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Languagecited by 71
  12. 2021Towards Natural Language Interfaces for Data Visualization: A Surveycited by 58
13 more
  1. 2024VisEval: A Benchmark for Data Visualization in the Era of Large Language Modelscited by 43
  2. 2024WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualizationcited by 38
  3. 2023Data Formulator: AI-powered Concept-driven Visualization Authoringcited by 31
  4. 2024LEVA: Using Large Language Models to Enhance Visual Analyticscited by 24
  5. 2024LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Executioncited by 20
  6. 2024Are LLMs ready for Visualization?cited by 16
  7. 2024DracoGPT: Extracting Visualization Design Preferences from Large Language Modelscited by 12
  8. 2022NL2INTERFACE: Interactive Visualization Interface Generation from Natural Language Queriescited by 8
  9. 2025Widget2Code: From Visual Widgets to UI Code via Multimodal LLMscited by 8
  10. 2022DMiner: Dashboard Design Mining and Recommendationcited by 4
  11. 2024UITrans: Seamless UI Translation from Android to HarmonyOScited by 1
  12. 2025ChartOptimiser: Task-driven Optimisation of Chart Designscited by 1
  13. 2025Urbanite: A Dataflow-Based Framework for Human-AI Interactive Alignment in Urban Visual Analyticscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SO-Bench: A Structural Output Evaluation of Multimodal LLMs
  2. 2026Beyond Static Charts: Can Language and Vision Language Models Generate Interactive Data Visualization Interfaces?
  3. 2024VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?
  4. 2024The Revolution of Multimodal Large Language Models: A Survey
  5. 2026ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation
  6. 2026From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs
  7. 2026Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages
  8. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping