# Type-Directed Synthesis of Visualizations from Natural Language Queries

arXiv: https://arxiv.org/abs/2209.01081 · HTML: https://mlsanity.com/paper/2209.01081/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2022.

## Built on by (most cited first, 5 of 5)

- [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023)
- [ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifiers](https://mlsanity.com/paper/2305.14591/) (2023)
- [Data Formulator: AI-powered Concept-driven Visualization Authoring](https://mlsanity.com/paper/2309.10094/) (2023)
- [Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way](https://mlsanity.com/paper/2408.16119/) (2024)
- [Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization](https://mlsanity.com/paper/2407.06129/) (2024)

## Abstract (arXiv)

We propose a new technique based on program synthesis for automatically generating visualizations from natural language queries. Our method parses the natural language query into a refinement type specification using the intents-and-slots paradigm and leverages type-directed synthesis to generate a set of visualization programs that are most likely to meet the user's intent. Our refinement type system captures useful hints present in the natural language query and allows the synthesis algorithm to reject visualizations that violate well-established design guidelines for the input data set. We have implemented our ideas in a tool called Graphy and evaluated it on NLVCorpus, which consists of 3 popular datasets and over 700 real-world natural language queries. Our experiments show that Graphy significantly outperforms state-of-the-art natural-language-based visualization tools, including transformer and rule-based ones.