# VegaChat: A Robust Framework for LLM-Based Chart Generation and Assessment

arXiv: https://arxiv.org/abs/2601.15385 · HTML: https://mlsanity.com/paper/2601.15385/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Builds on (most cited first, 10 of 10)

- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks](https://mlsanity.com/paper/2304.14979/) (2023)
- [Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations](https://mlsanity.com/paper/2110.00680/) (2021)
- [Visualization Generation with Large Language Models: An Evaluation](https://mlsanity.com/paper/2401.11255/) (2024)
- [IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking](https://mlsanity.com/paper/2410.07295/) (2024)
- [Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code](https://mlsanity.com/paper/2412.02764/) (2024)
- [nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow](https://mlsanity.com/paper/2502.05036/) (2025)
- [Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way](https://mlsanity.com/paper/2408.16119/) (2024)
- [Prompt4Vis: Prompting Large Language Models with Example Mining and Schema Filtering for Tabular Data Visualization](https://mlsanity.com/paper/2402.07909/) (2024)

## Built on by (most cited first, 1 of 1)

- [Agentic Authoring of Interactive Multiview Visualizations in Genomics](https://mlsanity.com/paper/2606.00370/) (2026)

## Abstract (arXiv)

Natural-language-to-visualization (NL2VIS) systems based on large language models (LLMs) have substantially improved the accessibility of data visualization. However, their further adoption is hindered by two coupled challenges: (i) the absence of standardized evaluation metrics makes it difficult to assess progress in the field and compare different approaches; and (ii) natural language descriptions are inherently underspecified, so multiple visualizations may be valid for the same query. To address these issues, we introduce VegaChat, a framework for generating, validating, and assessing declarative visualizations from natural language. We propose two complementary metrics: Spec Score, a deterministic metric that measures specification-level similarity without invoking an LLM, and Vision Score, a library-agnostic, image-based metric that leverages a multimodal LLM to assess chart similarity and prompt compliance. We evaluate VegaChat on the NLV Corpus and on the annotated subset of ChartLLM. VegaChat achieves near-zero rates of invalid or empty visualizations, while Spec Score and Vision Score exhibit strong correlation with human judgments (Pearson 0.65 and 0.71, respectively), indicating that the proposed metrics support consistent, cross-library comparison. The code and evaluation artifacts are available at https://zenodo.org/records/17062309.