A Review on Large Language Models for Visual Analytics
Not on the map. Why?
Abstract (arXiv)
This paper provides a comprehensive review of the integration of Large Language Models (LLMs) with visual analytics, addressing their foundational concepts, capabilities, and wide-ranging applications. It begins by outlining the theoretical underpinnings of visual analytics and the transformative potential of LLMs, specifically focusing on their roles in natural language understanding, natural language generation, dialogue systems, and text-to-media transformations. The review further investigates how the synergy between LLMs and visual analytics enhances data interpretation, visualization techniques, and interactive exploration capabilities. Key tools and platforms including LIDA, Chat2VIS, Julius AI, and Zoho Analytics, along with specialized multimodal models such as ChartLlama and CharXIV, are critically evaluated. The paper discusses their functionalities, strengths, and limitations in supporting data exploration, visualization enhancement, automated reporting, and insight extraction. The taxonomy of LLM tasks, ranging from natural language understanding (NLU), natural language generation (NLG), to dialogue systems and text-to-media transformations, is systematically explored. This review provides a SWOT analysis of integrating Large Language Models (LLMs) with visual analytics, highlighting strengths like accessibility and flexibility, weaknesses such as computational demands and biases, opportunities in multimodal integration and user collaboration, and threats including privacy concerns and skill degradation. It emphasizes addressing ethical considerations and methodological improvements for effective integration.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 72 of its 123 references that are in the map. First posted on arXiv 19 Mar 2025.
Builds on
72 of 123 references.
References · 72
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Visual Instruction Tuningcited by 7,869
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
38 more
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
- 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2023BloombergGPT: A Large Language Model for Financecited by 939
- 2023Instruction Tuning with GPT-4cited by 686
- 2022Towards Reasoning in Large Language Models: A Surveycited by 596
- 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
- 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
- 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
- 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 163
- 2021PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computationcited by 134
- 2023BuboGPT: Enabling Visual Grounding in Multi-Modal LLMscited by 114
- 2020AVLnet: Learning Audio-Visual Language Representations from Instructional Videoscited by 111
- 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 110
- 2023LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Modelscited by 82
- 2023ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Languagecited by 73
- 2024Hypothesis Generation with Large Language Modelscited by 71
- 2018Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Modelscited by 69
- 2023Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Modelscited by 67
- 2023Automating Customer Service using LangChain: Building custom open-source GPT Chatbot for organizationscited by 61
- 2021Towards Natural Language Interfaces for Data Visualization: A Surveycited by 59
- 2023Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Modelscited by 59
- 2024Koala: Key frame-conditioned long video-LLMcited by 57
- 2021CPM-2: Large-scale Cost-effective Pre-trained Language Modelscited by 52
- 2021Named Entity Recognition and Classification on Historical Documents: A Surveycited by 47
- 2023Visual cognition in multimodal large language modelscited by 38
- 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
- 2023How Do Analysts Understand and Verify AI-Assisted Data Analyses?cited by 26
- 2024Can Large Language Models Understand Context?cited by 26
50 most cited of 72 All 72 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.
- 2025Does visualization help AI understand data?cited by 3
- 2025Does visualization help AI understand data?24 Jul 2025