# Evaluating LLMs for Visualization Generation and Understanding

arXiv: https://arxiv.org/abs/2507.22890 · HTML: https://mlsanity.com/paper/2507.22890/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models](https://mlsanity.com/paper/2310.14566/) (2023)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [Chart-to-Text: A Large-Scale Benchmark for Chart Summarization](https://mlsanity.com/paper/2203.06486/) (2022)
- [ChartLlama: A Multimodal LLM for Chart Understanding and Generation](https://mlsanity.com/paper/2311.16483/) (2023)
- [UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model](https://mlsanity.com/paper/2310.05126/) (2023)
- [ChartBench: A Benchmark for Complex Visual Reasoning in Charts](https://mlsanity.com/paper/2312.15915/) (2023)
- [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023)
- [ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning](https://mlsanity.com/paper/2401.02384/) (2024)
- [NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries](https://mlsanity.com/paper/2008.10723/) (2020)
- [ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language](https://mlsanity.com/paper/2311.01920/) (2023)
- [SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs](https://mlsanity.com/paper/2308.03349/) (2023)
- [Towards Natural Language Interfaces for Data Visualization: A Survey](https://mlsanity.com/paper/2109.03506/) (2021)
- [Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Models](https://mlsanity.com/paper/2302.02094/) (2023)
- [Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networks](https://mlsanity.com/paper/1804.03126/) (2018)
- [VisEval: A Benchmark for Data Visualization in the Era of Large Language Models](https://mlsanity.com/paper/2407.00981/) (2024)
- [AI4VIS: Survey on Artificial Intelligence Approaches for Data Visualization](https://mlsanity.com/paper/2102.01330/) (2021)
- [Visualization Generation with Large Language Models: An Evaluation](https://mlsanity.com/paper/2401.11255/) (2024)
- [How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?](https://mlsanity.com/paper/2407.17291/) (2024)
- [Let the Chart Spark: Embedding Semantic Context into Chart with Text-to-Image Generative Model](https://mlsanity.com/paper/2304.14630/) (2023)
- [Are LLMs ready for Visualization?](https://mlsanity.com/paper/2403.06158/) (2024)

## Built on by (most cited first, 5 of 5)

- [CoDA: Agentic Systems for Collaborative Data Visualization](https://mlsanity.com/paper/2510.03194/) (2025)
- [SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents](https://mlsanity.com/paper/2603.29139/) (2026)
- [Liberating Logic in the Age of AI: Going Beyond Programming with Computational Thinking](https://mlsanity.com/paper/2511.17696/) (2025)
- [Evaluating Prompting Strategies for Chart Question Answering with Large Language Models](https://mlsanity.com/paper/2603.22288/) (2026)
- [Curriculum as Code: An AI-Assisted Architecture for Instructional Design in STEM Education](https://mlsanity.com/paper/2608.07364/) (2026)

## Abstract (arXiv)

Information Visualization has been utilized to gain insights from complex data. In recent times, Large Language models (LLMs) have performed very well in many tasks. In this paper, we showcase the capabilities of different popular LLMs to generate code for visualization based on simple prompts. We also analyze the power of LLMs to understand some common visualizations by answering questions. Our study shows that LLMs could generate code for some simpler visualizations such as bar and pie charts. Moreover, they could answer simple questions about visualizations. However, LLMs also have several limitations. For example, some of them had difficulty generating complex visualizations, such as violin plot. LLMs also made errors in answering some questions about visualizations, for example, identifying relationships between close boundaries and determining lengths of shapes. We believe that our insights can be used to improve both LLMs and Information Visualization systems.