# Facilitating Conversational Interaction in Natural Language Interfaces for Visualization

arXiv: https://arxiv.org/abs/2207.00189 · HTML: https://mlsanity.com/paper/2207.00189/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2022.

## Built on by (most cited first, 12 of 12)

- [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023)
- [Natural Language Interfaces for Tabular Data Querying and Visualization: A Survey](https://mlsanity.com/paper/2310.17894/) (2023)
- [AVA: Towards Autonomous Visualization Agents through Visual Perception-Driven Decision-Making](https://mlsanity.com/paper/2312.04494/) (2023)
- [Data Formulator 2: Iterative Creation of Data Visualizations, with AI Transforming Data Along the Way](https://mlsanity.com/paper/2408.16119/) (2024)
- [XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis](https://mlsanity.com/paper/2301.10385/) (2023)
- [Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities](https://mlsanity.com/paper/2304.01311/) (2023)
- [LinkQ: An LLM-Assisted Visual Interface for Knowledge Graph Question-Answering](https://mlsanity.com/paper/2406.06621/) (2024)
- [Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions](https://mlsanity.com/paper/2409.19747/) (2024)
- [InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs](https://mlsanity.com/paper/2312.09672/) (2023)
- [LLM-Assisted Visual Analytics: Opportunities and Challenges](https://mlsanity.com/paper/2409.02691/) (2024)
- [Augmenting a Large Language Model with a Combination of Text and Visual Data for Conversational Visualization of Global Geospatial Data](https://mlsanity.com/paper/2501.09521/) (2025)
- [The Role of Visualization in LLM-Assisted Knowledge Graph Systems: Effects on User Trust, Exploration, and Workflows](https://mlsanity.com/paper/2505.21512/) (2025)

## Abstract (arXiv)

Natural language (NL) toolkits enable visualization developers, who may not have a background in natural language processing (NLP), to create natural language interfaces (NLIs) for end-users to flexibly specify and interact with visualizations. However, these toolkits currently only support one-off utterances, with minimal capability to facilitate a multi-turn dialog between the user and the system. Developing NLIs with such conversational interaction capabilities remains a challenging task, requiring implementations of low-level NLP techniques to process a new query as an intent to follow-up on an older query. We extend an existing Python-based toolkit, NL4DV, that processes an NL query about a tabular dataset and returns an analytic specification containing data attributes, analytic tasks, and relevant visualizations, modeled as a JSON object. Specifically, NL4DV now enables developers to facilitate multiple simultaneous conversations about a dataset and resolve associated ambiguities, augmenting new conversational information into the output JSON object. We demonstrate these capabilities through three examples: (1) an NLI to learn aspects of the Vega-Lite grammar, (2) a mind mapping application to create free-flowing conversations, and (3) a chatbot to answer questions and resolve ambiguities.