# Visualization Generation with Large Language Models: An Evaluation

arXiv: https://arxiv.org/abs/2401.11255 · HTML: https://mlsanity.com/paper/2401.11255/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2024.

## Built on by (most cited first, 29 of 29)

- [VisEval: A Benchmark for Data Visualization in the Era of Large Language Models](https://mlsanity.com/paper/2407.00981/) (2024)
- [TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understanding](https://mlsanity.com/paper/2502.19400/) (2025)
- [nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow](https://mlsanity.com/paper/2502.05036/) (2025)
- [Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Framework](https://mlsanity.com/paper/2506.02454/) (2025)
- [LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Execution](https://mlsanity.com/paper/2411.05651/) (2024)
- [AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy](https://mlsanity.com/paper/2505.20538/) (2025)
- [Visualization Literacy of Multimodal Large Language Models: A Comparative Study](https://mlsanity.com/paper/2407.10996/) (2024)
- [VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation](https://mlsanity.com/paper/2506.13326/) (2025)
- [DataLab: A Unified Platform for LLM-Powered Business Intelligence](https://mlsanity.com/paper/2412.02205/) (2024)
- [Automated Visualization Code Synthesis via Multi-Path Reasoning and Feedback-Driven Optimization](https://mlsanity.com/paper/2502.11140/) (2025)
- [See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMs](https://mlsanity.com/paper/2504.09809/) (2025)
- [Charts-of-Thought: Enhancing LLM Visualization Literacy Through Structured Data Extraction](https://mlsanity.com/paper/2508.04842/) (2025)
- [An Evaluation-Centric Paradigm for Scientific Visualization Agents](https://mlsanity.com/paper/2509.15160/) (2025)
- [Exploring Multimodal Prompt for Visualization Authoring with Large Language Models](https://mlsanity.com/paper/2504.13700/) (2025)
- [Evaluating LLMs for Visualization Generation and Understanding](https://mlsanity.com/paper/2507.22890/) (2025)
- [Charting the Future: Using Chart Question-Answering for Scalable Evaluation of LLM-Driven Data Visualizations](https://mlsanity.com/paper/2409.18764/) (2024)
- [DashChat: Interactive Authoring of Performance Dashboard Design Prototypes through Conversation with LLM-Powered Agent](https://mlsanity.com/paper/2504.12865/) (2025)
- [Do Language Model Agents Align with Humans in Rating Visualizations? An Empirical Study](https://mlsanity.com/paper/2505.06702/) (2025)
- [Do LLMs Understand Collaborative Signals? Diagnosis and Repair](https://mlsanity.com/paper/2505.20730/) (2025)
- [Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach](https://mlsanity.com/paper/2506.06175/) (2025)
- [VegaChat: A Robust Framework for LLM-Based Chart Generation and Assessment](https://mlsanity.com/paper/2601.15385/) (2026)
- [Can LLMs Generate Visualizations with Dataless Prompts?](https://mlsanity.com/paper/2406.17805/) (2024)
- [Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization](https://mlsanity.com/paper/2407.06129/) (2024)
- [Do Text-to-Vis Benchmarks Test Real Use of Visualisations?](https://mlsanity.com/paper/2407.19726/) (2024)
- [Evaluating LLMs for Visualization Tasks](https://mlsanity.com/paper/2506.10996/) (2025)
- [From Static to Interactive: Authoring Interactive Visualizations via Natural Language](https://mlsanity.com/paper/2601.17736/) (2026)
- [Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models](https://mlsanity.com/paper/2609.22097/) (2026)
- [Where LLMs Fail with Visualization DSLs](https://mlsanity.com/paper/2610.01873/) (2026)
- [Zero-Shot Visualization: Exploring Text Corpora with User-Prompted Axes](https://mlsanity.com/paper/2610.06889/) (2026)

## Abstract (arXiv)

The frequent need for analysts to create visualizations to derive insights from data has driven extensive research into the generation of natural Language to Visualization (NL2VIS). While recent progress in large language models (LLMs) suggests their potential to effectively support NL2VIS tasks, existing studies lack a systematic investigation into the performance of different LLMs under various prompt strategies. This paper addresses this gap and contributes a crucial baseline evaluation of LLMs' capabilities in generating visualization specifications of NL2VIS tasks. Our evaluation utilizes the nvBench dataset, employing six representative LLMs and eight distinct prompt strategies to evaluate their performance in generating six target chart types using the Vega-Lite visualization specification. We assess model performance with multiple metrics, including vis accuracy, validity and legality. Our results reveal substantial performance disparities across prompt strategies, chart types, and LLMs. Furthermore, based on the evaluation results, we uncover several counterintuitive behaviors across these dimensions, and propose directions for enhancing the NL2VIS benchmark to better support future NL2VIS research.