# Captioning Visualizations with Large Language Models (CVLLM): A Tutorial

arXiv: https://arxiv.org/abs/2406.19512 · HTML: https://mlsanity.com/paper/2406.19512/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [NExT-GPT: Any-to-Any Multimodal LLM](https://mlsanity.com/paper/2309.05519/) (2023)
- [VisText: A Benchmark for Semantically Rich Chart Captioning](https://mlsanity.com/paper/2307.05356/) (2023)
- [Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning](https://mlsanity.com/paper/2312.10160/) (2023)
- [Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content](https://mlsanity.com/paper/2110.04406/) (2021)
- [ChartThinker: A Contextual Chain-of-Thought Approach to Optimized Chart Summarization](https://mlsanity.com/paper/2403.11236/) (2024)
- [Why More Text is (Often) Better: Themes from Reader Preferences for Integration of Charts and Text](https://mlsanity.com/paper/2209.10789/) (2022)

## Abstract (arXiv)

Automatically captioning visualizations is not new, but recent advances in large language models(LLMs) open exciting new possibilities. In this tutorial, after providing a brief review of Information Visualization (InfoVis) principles and past work in captioning, we introduce neural models and the transformer architecture used in generic LLMs. We then discuss their recent applications in InfoVis, with a focus on captioning. Additionally, we explore promising future directions in this field.