# Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text

arXiv: https://arxiv.org/abs/2507.19969 · HTML: https://mlsanity.com/paper/2507.19969/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2025.

## Built on by (most cited first, 12 of 12)

- [VisCoder2: Building Multi-Language Visualization Coding Agents](https://mlsanity.com/paper/2510.23642/) (2025)
- [OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward](https://mlsanity.com/paper/2604.05514/) (2026)
- [DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards](https://mlsanity.com/paper/2508.17398/) (2025)
- [DATAREEL: Automated Data-Driven Video Story Generation with Animations](https://mlsanity.com/paper/2604.25220/) (2026)
- [PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning](https://mlsanity.com/paper/2605.19382/) (2026)
- [Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards](https://mlsanity.com/paper/2607.04727/) (2026)
- [AnnoBench: A Benchmark for Visualization Annotation Generation](https://mlsanity.com/paper/2607.25911/) (2026)
- [Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices](https://mlsanity.com/paper/2510.07545/) (2025)
- [Business Utility of Large Language Models as Exploratory Data Analysis Agents](https://mlsanity.com/paper/2606.00051/) (2026)
- [Chart Deception in Vision-Language Models: From Vulnerability to Mitigation](https://mlsanity.com/paper/2607.22600/) (2026)
- [ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation](https://mlsanity.com/paper/2608.03464/) (2026)
- [DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?](https://mlsanity.com/paper/2608.26757/) (2026)

## Abstract (arXiv)

Automated data visualization plays a crucial role in simplifying data interpretation, enhancing decision-making, and improving efficiency. While large language models (LLMs) have shown promise in generating visualizations from natural language, the absence of comprehensive benchmarks limits the rigorous evaluation of their capabilities. We introduce Text2Vis, a benchmark designed to assess text-to-visualization models, covering 20+ chart types and diverse data science queries, including trend analysis, correlation, outlier detection, and predictive analytics. It comprises 1,985 samples, each with a data table, natural language query, short answer, visualization code, and annotated charts. The queries involve complex reasoning, conversational turns, and dynamic data retrieval. We benchmark 11 open-source and closed-source models, revealing significant performance gaps, highlighting key challenges, and offering insights for future advancements. To close this gap, we propose the first cross-modal actor-critic agentic framework that jointly refines the textual answer and visualization code, increasing GPT-4o`s pass rate from 26% to 42% over the direct approach and improving chart quality. We also introduce an automated LLM-based evaluation framework that enables scalable assessment across thousands of samples without human annotation, measuring answer correctness, code execution success, visualization readability, and chart accuracy. We release Text2Vis at https://github.com/vis-nlp/Text2Vis.