# Using Large Language Models to Generate Engaging Captions for Data Visualizations

arXiv: https://arxiv.org/abs/2212.14047 · HTML: https://mlsanity.com/paper/2212.14047/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 7 references that are in the map.
- First posted on arXiv 27 Dec 2022.

## Builds on (most cited first, 2 of 2)

- [From Show to Tell: A Survey on Deep Learning-based Image Captioning](https://mlsanity.com/paper/2107.06912/) (2021)
- [Towards Natural Language Interfaces for Data Visualization: A Survey](https://mlsanity.com/paper/2109.03506/) (2021)

## Built on by (most cited first, 5 of 5)

- [Benchmarking Visual Language Models on Standardized Visualization Literacy Tests](https://mlsanity.com/paper/2503.16632/) (2025)
- [Algorithmic Ghost in the Research Shell: Large Language Models and Academic Knowledge Creation in Management Research](https://mlsanity.com/paper/2303.07304/) (2023)
- [Charts-of-Thought: Enhancing LLM Visualization Literacy Through Structured Data Extraction](https://mlsanity.com/paper/2508.04842/) (2025)
- [A Review on Large Language Models for Visual Analytics](https://mlsanity.com/paper/2503.15176/) (2025)
- [Automated Visualization Makeovers with LLMs](https://mlsanity.com/paper/2508.05637/) (2025)

## Abstract (arXiv)

Creating compelling captions for data visualizations has been a longstanding challenge. Visualization researchers are typically untrained in journalistic reporting and hence the captions that are placed below data visualizations tend to be not overly engaging and rather just stick to basic observations about the data. In this work we explore the opportunities offered by the newly emerging crop of large language models (LLM) which use sophisticated deep learning technology to produce human-like prose. We ask, can these powerful software devices be purposed to produce engaging captions for generic data visualizations like a scatterplot. It turns out that the key challenge lies in designing the most effective prompt for the LLM, a task called prompt engineering. We report on first experiments using the popular LLM GPT-3 and deliver some promising results.