mlsanityData to 7 Oct 2026

VisEval: A Benchmark for Data Visualization in the Era of Large Language Models

arXiv 2407.00981 ↗First posted 1 Jul 2024cs.HC cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Translating natural language to visualization (NL2VIS) has shown great promise for visual data analysis, but it remains a challenging task that requires multiple low-level implementations, such as natural language processing and visualization design. Recent advancements in pre-trained large language models (LLMs) are opening new avenues for generating visualizations from natural language. However, the lack of a comprehensive and reliable benchmark hinders our understanding of LLMs' capabilities in visualization generation. In this paper, we address this gap by proposing a new NL2VIS benchmark called VisEval. Firstly, we introduce a high-quality and large-scale dataset. This dataset includes 2,524 representative queries covering 146 databases, paired with accurately labeled ground truths. Secondly, we advocate for a comprehensive automated evaluation methodology covering multiple dimensions, including validity, legality, and readability. By systematically scanning for potential issues with a number of heterogeneous checkers, VisEval provides reliable and trustworthy evaluation outcomes. We run VisEval on a series of state-of-the-art LLMs. Our evaluation reveals prevalent challenges and delivers essential insights for future advancements.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 50 of its 78 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

50 of 78 references.

References · 50

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  12. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
38 more
  1. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  2. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  3. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  4. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  5. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 158
  6. 2023Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Studycited by 116
  7. 2023LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Modelscited by 80
  8. 2020NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queriescited by 73
  9. 2023ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Languagecited by 71
  10. 2023Is GPT-4 a Good Data Analyst?cited by 69
  11. 2023MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Taskscited by 66
  12. 2021Towards Natural Language Interfaces for Data Visualization: A Surveycited by 58
  13. 2023Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Modelscited by 58
  14. 2018Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networkscited by 47
  15. 2023Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluationcited by 46
  16. 2018VizML: A Machine Learning Approach to Visualization Recommendationcited by 38
  17. 2021Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizationscited by 31
  18. 2023ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summariescited by 30
  19. 2024Benchmarking Data Science Agentscited by 30
  20. 2019VizNet: Towards A Large-Scale Visualization Learning and Benchmarking Repositorycited by 28
  21. 2024Visualization Generation with Large Language Models: An Evaluationcited by 28
  22. 2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Surveycited by 27
  23. 2019FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow Systemcited by 25
  24. 2023How Do Analysts Understand and Verify AI-Assisted Data Analyses?cited by 25
  25. 2021VizLinter: A Linter and Fixer Framework for Data Visualizationcited by 20
  26. 2023How Good is ChatGPT in Giving Advice on Your Visualization Design?cited by 20
  27. 2022Misinformed by Visualization: What Do We Learn From Misinformative Visualizations?cited by 19
  28. 2023Beyond Generating Code: Evaluating GPT on a Data Visualization Coursecited by 16
  29. 2024Are LLMs ready for Visualization?cited by 16
  30. 2024Prompt4Vis: Prompting Large Language Models with Example Mining and Schema Filtering for Tabular Data Visualizationcited by 14
  31. 2024Vi(E)va LLM! A Conceptual Stack for Evaluating and Interpreting Generative AI-based Visualizationscited by 10
  32. 2020Quda: Natural Language Queries for Visual Data Analyticscited by 9
  33. 2024Leveraging Foundation Models for Crafting Narrative Visualization: A Surveycited by 9
  34. 2023Conversational AI Threads for Visualizing Multidimensional Datasetscited by 8
  35. 2023Chat2VIS: Fine-Tuning Data Visualisations using Multilingual Natural Language Text and Pre-Trained Large Language Modelscited by 6
  36. 2024Assessing the Aesthetic Evaluation Capabilities of GPT-4 with Vision: Insights from Group and Individual Assessmentscited by 4
  37. 2020MobileVisFixer: Tailoring Web Visualizations for Mobile Phones Leveraging an Explainable Reinforcement Learning Frameworkcited by 3
  38. 2021Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparisoncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 21; 2026: 20.

43 papers.

Sort:

By area:

  1. 2025Benchmarking Visual Language Models on Standardized Visualization Literacy Testscited by 20
  2. 2025NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splattingcited by 19
  3. 2025CoDA: Agentic Systems for Collaborative Data Visualizationcited by 14
  4. 2025AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomycited by 13
  5. 2026SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agentscited by 9
  6. 2025VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generationcited by 8
  7. 2025DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoningcited by 8
  8. 2024DataLab: A Unified Platform for LLM-Powered Business Intelligencecited by 7
  9. 2025See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMscited by 7
  10. 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizationscited by 7
  11. 2025Paper2SysArch: Structure-Constrained System Architecture Generation from Scientific Paperscited by 7
  12. 2024Unveiling and Mitigating Bias in Large Language Model Recommendations: A Path to Fairnesscited by 6
31 more
  1. 2025Stop Misusing t-SNE and UMAP for Visual Analyticscited by 6
  2. 2025An Evaluation-Centric Paradigm for Scientific Visualization Agentscited by 6
  3. 2025Exploring Multimodal Prompt for Visualization Authoring with Large Language Modelscited by 5
  4. 2025Evaluating LLMs for Visualization Generation and Understandingcited by 5
  5. 2025IntuiTF: MLLM-Guided Transfer Function Optimization for Direct Volume Renderingcited by 4
  6. 2026DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Sciencecited by 3
  7. 2026Exploring LLM Agent Designs and Interaction Modalities for Scientific Visualizationcited by 2
  8. 2026Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Taskscited by 2
  9. 2026ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Modelscited by 2
  10. 2025Augmenting a Large Language Model with a Combination of Text and Visual Data for Conversational Visualization of Global Geospatial Datacited by 1
  11. 2025Do Language Model Agents Align with Humans in Rating Visualizations? An Empirical Studycited by 1
  12. 2025A Generative AI System for Biomedical Data Discovery with Grammar-Based Visualizationscited by 1
  13. 2025YAC: Bridging Natural Language and Interactive Visual Exploration with Generative AI for Biomedical Data Discoverycited by 1
  14. 2025MedInsightBench: Evaluating Medical Analytics Agents Through Multi-Step Insight Discovery in Multimodal Medical Datacited by 1
  15. 2026Generating Statistical Charts with Validation-Driven LLM Workflowscited by 1
  16. 2026SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualizationcited by 1
  17. 2026AnnoBench: A Benchmark for Visualization Annotation Generationcited by 1
  18. 2025VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agentscited by 0
  19. 2025Automated Visualization Makeovers with LLMscited by 0
  20. 2026Human-Data Interaction, Exploration, and Visualization in the AI Era: Challenges and Opportunitiescited by 0
  21. 2026Evaluating Prompting Strategies for Chart Question Answering with Large Language Modelscited by 0
  22. 2026RedParrot: Accelerating NL-to-DSL for Business Analytics via Query Semantic Cachingcited by 0
  23. 2026Agentic Authoring of Interactive Multiview Visualizations in Genomicscited by 0
  24. 2026MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Constructioncited by 0
  25. 2026Harnessing the Wisdom of LLM Crowds through Complementarity-Driven Iterative Collaborationcited by 0
  26. 2026ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generationcited by 0
  27. 2026Exploring Agentic Approaches for Data Issue Detection and Repair in AI-Assisted Visualizationcited by 0
  28. 2026VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queriescited by 0
  29. 2026Lexara-RF: Reference-Free Metrics for Evaluating Conversational Visual Analytics Agentscited by 0
  30. 2026Token Signatures of Code: Comparing Coding Behaviors Across Large Language Modelscited by 0
  31. 2026Where LLMs Fail with Visualization DSLscited by 0