# Generative AI for Visualization: State of the Art and Future Directions

arXiv: https://arxiv.org/abs/2404.18144 · HTML: https://mlsanity.com/paper/2404.18144/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 137 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 30 of 68)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [Unifying Vision-and-Language Tasks via Text Generation](https://mlsanity.com/paper/2102.02779/) (2021)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [TabLLM: Few-shot Classification of Tabular Data with Large Language Models](https://mlsanity.com/paper/2210.10723/) (2022)
- [A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2001.06937/) (2020)
- [Data Synthesis based on Generative Adversarial Networks](https://mlsanity.com/paper/1806.03384/) (2018)
- [Sketch-Guided Text-to-Image Diffusion Models](https://mlsanity.com/paper/2211.13752/) (2022)
- [Chart-to-Text: A Large-Scale Benchmark for Chart Summarization](https://mlsanity.com/paper/2203.06486/) (2022)
- [UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning](https://mlsanity.com/paper/2305.14761/) (2023)
- [Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow](https://mlsanity.com/paper/1802.10123/) (2018)
- [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023)
- [NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries](https://mlsanity.com/paper/2008.10723/) (2020)
- [ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language](https://mlsanity.com/paper/2311.01920/) (2023)
- [Towards Natural Language Interfaces for Data Visualization: A Survey](https://mlsanity.com/paper/2109.03506/) (2021)
- [Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networks](https://mlsanity.com/paper/1804.03126/) (2018)
- [V4d: voxel for 4d novel view synthesis](https://mlsanity.com/paper/2205.14332/) (2022)
- [VizML: A Machine Learning Approach to Visualization Recommendation](https://mlsanity.com/paper/1808.04819/) (2018)
- [AI4VIS: Survey on Artificial Intelligence Approaches for Data Visualization](https://mlsanity.com/paper/2102.01330/) (2021)
- [GNN-Surrogate: A Hierarchical and Adaptive Graph Neural Network for Parameter Space Exploration of Unstructured-Mesh Ocean Simulations](https://mlsanity.com/paper/2202.08956/) (2022)
- [DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization](https://mlsanity.com/paper/2204.06504/) (2022)
- [A Generative Model for Volume Rendering](https://mlsanity.com/paper/1710.09545/) (2017)
- [Text-to-Viz: Automatic Generation of Infographics from Proportion-Related Natural Language Statements](https://mlsanity.com/paper/1907.09091/) (2019)
- [Foundation Models Meet Visualizations: Challenges and Opportunities](https://mlsanity.com/paper/2310.05771/) (2023)

## Built on by (most cited first, 23 of 23)

- [AFlow: Automating Agentic Workflow Generation](https://mlsanity.com/paper/2410.10762/) (2024)
- [TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems](https://mlsanity.com/paper/2506.04133/) (2025)
- [ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering](https://mlsanity.com/paper/2405.07001/) (2024)
- [Self-Supervised Prompt Optimization](https://mlsanity.com/paper/2502.06855/) (2025)
- [Are Large Language Models Good Statisticians?](https://mlsanity.com/paper/2406.07815/) (2024)
- [HAIChart: Human and AI Paired Visualization System](https://mlsanity.com/paper/2406.11033/) (2024)
- [Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executors](https://mlsanity.com/paper/2504.04785/) (2025)
- [InterChat: Enhancing Generative Visual Analytics using Multimodal Interactions](https://mlsanity.com/paper/2503.04110/) (2025)
- [Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems](https://mlsanity.com/paper/2505.19101/) (2025)
- [ParaView-MCP: An Autonomous Visualization Agent with Direct Tool Use](https://mlsanity.com/paper/2505.07064/) (2025)
- [ChartInsighter: An Approach for Mitigating Hallucination in Time-series Chart Summary Generation with A Benchmark Dataset](https://mlsanity.com/paper/2501.09349/) (2025)
- [Exploring Multimodal Prompt for Visualization Authoring with Large Language Models](https://mlsanity.com/paper/2504.13700/) (2025)
- [ChartCards: A Chart-Metadata Generation Framework for Multi-Task Chart Understanding](https://mlsanity.com/paper/2505.15046/) (2025)
- [LLM-Assisted Visual Analytics: Opportunities and Challenges](https://mlsanity.com/paper/2409.02691/) (2024)
- [Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Framework](https://mlsanity.com/paper/2606.29808/) (2026)
- [DashChat: Interactive Authoring of Performance Dashboard Design Prototypes through Conversation with LLM-Powered Agent](https://mlsanity.com/paper/2504.12865/) (2025)
- [The Digital Mirror: Gender Bias and Occupational Stereotypes in AI-Generated Images](https://mlsanity.com/paper/2510.08628/) (2025)
- [Elementary School Students' and Teachers' Perceptions Towards Creative Mathematical Writing with Generative AI](https://mlsanity.com/paper/2409.06723/) (2024)
- [AKRMap: Adaptive Kernel Regression for Trustworthy Visualization of Cross-Modal Embeddings](https://mlsanity.com/paper/2505.14664/) (2025)
- [Portal UX Agent -- A Plug-and-Play Engine for Rendering UIs from Natural Language Specifications](https://mlsanity.com/paper/2511.00843/) (2025)
- [CycleChart: A Unified Consistency-Based Learning Framework for Bidirectional Chart Understanding and Generation](https://mlsanity.com/paper/2512.19173/) (2025)
- [Human-Data Interaction, Exploration, and Visualization in the AI Era: Challenges and Opportunities](https://mlsanity.com/paper/2603.05542/) (2026)
- [Beauty in the Eye of AI: Aligning LLMs and Vision Models with Human Aesthetics in Network Visualization](https://mlsanity.com/paper/2604.03417/) (2026)

## Abstract (arXiv)

Generative AI (GenAI) has witnessed remarkable progress in recent years and demonstrated impressive performance in various generation tasks in different domains such as computer vision and computational design. Many researchers have attempted to integrate GenAI into visualization framework, leveraging the superior generative capacity for different operations. Concurrently, recent major breakthroughs in GenAI like diffusion model and large language model have also drastically increase the potential of GenAI4VIS. From a technical perspective, this paper looks back on previous visualization studies leveraging GenAI and discusses the challenges and opportunities for future research. Specifically, we cover the applications of different types of GenAI methods including sequence, tabular, spatial and graph generation techniques for different tasks of visualization which we summarize into four major stages: data enhancement, visual mapping generation, stylization and interaction. For each specific visualization sub-task, we illustrate the typical data and concrete GenAI algorithms, aiming to provide in-depth understanding of the state-of-the-art GenAI4VIS techniques and their limitations. Furthermore, based on the survey, we discuss three major aspects of challenges and research opportunities including evaluation, dataset, and the gap between end-to-end GenAI and generative algorithms. By summarizing different generation algorithms, their current applications and limitations, this paper endeavors to provide useful insights for future GenAI4VIS research.