# Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations

arXiv: https://arxiv.org/abs/2110.00680 · HTML: https://mlsanity.com/paper/2110.00680/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2021.

## Built on by (most cited first, 30 of 31)

- [Synchromesh: Reliable code generation from pre-trained language models](https://mlsanity.com/paper/2201.11227/) (2022)
- [VisEval: A Benchmark for Data Visualization in the Era of Large Language Models](https://mlsanity.com/paper/2407.00981/) (2024)
- [Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content](https://mlsanity.com/paper/2110.04406/) (2021)
- [Data Formulator: AI-powered Concept-driven Visualization Authoring](https://mlsanity.com/paper/2309.10094/) (2023)
- [IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking](https://mlsanity.com/paper/2410.07295/) (2024)
- [nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task](https://mlsanity.com/paper/2112.12926/) (2021)
- [Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey](https://mlsanity.com/paper/2310.17894/) (2023)
- [Chart Question Answering: State of the Art and Future Directions](https://mlsanity.com/paper/2205.03966/) (2022)
- [XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis](https://mlsanity.com/paper/2301.10385/) (2023)
- [Vi(E)va LLM! A Conceptual Stack for Evaluating and Interpreting Generative AI-based Visualizations](https://mlsanity.com/paper/2402.02167/) (2024)
- [VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation](https://mlsanity.com/paper/2506.13326/) (2025)
- [Conversational AI Threads for Visualizing Multidimensional Datasets](https://mlsanity.com/paper/2311.05590/) (2023)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)
- [Exploring Multimodal Prompt for Visualization Authoring with Large Language Models](https://mlsanity.com/paper/2504.13700/) (2025)
- [Automatic Data Visualization Generation from Chinese Natural Language Questions](https://mlsanity.com/paper/2309.07650/) (2023)
- [V-RECS, a Low-Cost LLM4VIS Recommender with Explanations, Captioning and Suggestions](https://mlsanity.com/paper/2406.15259/) (2024)
- [MultiVis-Agent: A Multi-Agent Framework with Logic Rules for Reliable and Comprehensive Cross-Modal Data Visualization](https://mlsanity.com/paper/2601.18320/) (2026)
- [Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization](https://mlsanity.com/paper/2601.04582/) (2026)
- [MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model](https://mlsanity.com/paper/2501.01014/) (2025)
- [Capturing Visualization Design Rationale](https://mlsanity.com/paper/2506.16571/) (2025)
- [VegaChat: A Robust Framework for LLM-Based Chart Generation and Assessment](https://mlsanity.com/paper/2601.15385/) (2026)
- [Lexara: A User-Centered Toolkit for Evaluating Large Language Models for Conversational Visual Analytics](https://mlsanity.com/paper/2603.05832/) (2026)
- [Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization](https://mlsanity.com/paper/2407.06129/) (2024)
- [Do Text-to-Vis Benchmarks Test Real Use of Visualisations?](https://mlsanity.com/paper/2407.19726/) (2024)
- [Macro-Queries: An Exploration into Guided Chart Generation from High Level Prompts](https://mlsanity.com/paper/2408.12726/) (2024)
- [NOVAID: Natural-language Observability Visualization Assistant for ITOps Dashboard Widget Generation](https://mlsanity.com/paper/2601.11531/) (2025)
- [Agentic Authoring of Interactive Multiview Visualizations in Genomics](https://mlsanity.com/paper/2606.00370/) (2026)
- [Crystalis: Progressive Nucleation and Semantic Annealing for Coordinated Multi-View Visualization Generation](https://mlsanity.com/paper/2607.24766/) (2026)
- [VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queries](https://mlsanity.com/paper/2609.15182/) (2026)
- [Lexara-RF: Reference-Free Metrics for Evaluating Conversational Visual Analytics Agents](https://mlsanity.com/paper/2609.17842/) (2026)

## Abstract (arXiv)

Natural language interfaces (NLIs) for data visualization are becoming increasingly popular both in academic research and in commercial software. Yet, there is a lack of empirical understanding of how people specify visualizations through natural language. To bridge this gap, we conducted an online study with 102 participants. We showed participants a series of ten visualizations for a given dataset and asked them to provide utterances they would pose to generate the displayed charts. The curated list of utterances generated from the study is provided below. This corpus of utterances can be used to evaluate existing NLIs for data visualization as well as for creating new systems and models to generate visualizations from natural language utterances.