# Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration

arXiv: https://arxiv.org/abs/2309.15723 · HTML: https://mlsanity.com/paper/2309.15723/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 156 references that are in the map.
- First posted on arXiv 27 Sep 2023.

## Builds on (most cited first, 30 of 55)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [RuleMatrix: Visualizing and Understanding Classifiers with Rules](https://mlsanity.com/paper/1807.06228/) (2018)
- [VizML: A Machine Learning Approach to Visualization Recommendation](https://mlsanity.com/paper/1808.04819/) (2018)
- [AI4VIS: Survey on Artificial Intelligence Approaches for Data Visualization](https://mlsanity.com/paper/2102.01330/) (2021)
- [Text-to-Viz: Automatic Generation of Infographics from Proportion-Related Natural Language Statements](https://mlsanity.com/paper/1907.09091/) (2019)
- [Calliope: Automatic Visual Data Story Generation from a Spreadsheet](https://mlsanity.com/paper/2010.09975/) (2020)
- [A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization](https://mlsanity.com/paper/2012.00467/) (2020)
- [Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values](https://mlsanity.com/paper/2206.15465/) (2022)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articles](https://mlsanity.com/paper/2308.04076/) (2023)
- [Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop](https://mlsanity.com/paper/2101.04296/) (2021)
- [Towards Automated Infographic Design: Deep Learning-based Auto-Extraction of Extensible Timeline](https://mlsanity.com/paper/1907.13550/) (2019)
- [VizLinter: A Linter and Fixer Framework for Data Visualization](https://mlsanity.com/paper/2108.10299/) (2021)
- [InfoColorizer: Interactive Recommendation of Color Palettes for Infographics](https://mlsanity.com/paper/2102.02041/) (2021)
- [Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work](https://mlsanity.com/paper/2203.11085/) (2022)
- [Retrieve-Then-Adapt: Example-based Automatic Generation for Proportion-related Infographics](https://mlsanity.com/paper/2008.01177/) (2020)
- [Data Player: Automatic Generation of Data Videos with Narration-Animation Interplay](https://mlsanity.com/paper/2308.04703/) (2023)
- [VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking](https://mlsanity.com/paper/2001.07876/) (2020)
- [Supporting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis](https://mlsanity.com/paper/2102.03702/) (2021)
- [Structure-aware Visualization Retrieval](https://mlsanity.com/paper/2202.05960/) (2022)
- [Erato: Cooperative Data Story Editing via Fact Interpolation](https://mlsanity.com/paper/2209.02529/) (2022)
- [Sporthesia: Augmenting Sports Videos Using Natural Language](https://mlsanity.com/paper/2209.03434/) (2022)
- [InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebooks](https://mlsanity.com/paper/2307.07922/) (2023)
- [ChartStory: Automated Partitioning, Layout, and Captioning of Charts into Comic-Style Narratives](https://mlsanity.com/paper/2103.03996/) (2021)

## Built on by (most cited first, 13 of 13)

- [From Data to Story: Towards Automatic Animated Data Video Creation with LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2408.03876/) (2024)
- [Deconstructing Human-AI Collaboration: Agency, Interaction, and Adaptation](https://mlsanity.com/paper/2404.12056/) (2024)
- [DATAWEAVER: Authoring Data-Driven Narratives through the Integrated Composition of Visualization and Text](https://mlsanity.com/paper/2503.22946/) (2025)
- [TongSIM: A General Platform for Simulating Intelligent Machines](https://mlsanity.com/paper/2512.20206/) (2025)
- [ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMs](https://mlsanity.com/paper/2502.02329/) (2025)
- [MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model](https://mlsanity.com/paper/2501.01014/) (2025)
- [Reflection on Data Storytelling Tools in the Generative AI Era from the Human-AI Collaboration Perspective](https://mlsanity.com/paper/2503.02631/) (2025)
- [A Review on Large Language Models for Visual Analytics](https://mlsanity.com/paper/2503.15176/) (2025)
- [DashChat: Interactive Authoring of Performance Dashboard Design Prototypes through Conversation with LLM-Powered Agent](https://mlsanity.com/paper/2504.12865/) (2025)
- [Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Making](https://mlsanity.com/paper/2509.19643/) (2025)
- [AnnoBench: A Benchmark for Visualization Annotation Generation](https://mlsanity.com/paper/2607.25911/) (2026)
- [Human-Data Interaction, Exploration, and Visualization in the AI Era: Challenges and Opportunities](https://mlsanity.com/paper/2603.05542/) (2026)
- [Data storytelling meets interpretable machine learning: Decoding AI decisions for non-experts without revealing sensitive data and model details](https://mlsanity.com/paper/2609.15722/) (2026)

## Abstract (arXiv)

Data storytelling is powerful for communicating data insights, but it requires diverse skills and considerable effort from human creators. Recent research has widely explored the potential for artificial intelligence (AI) to support and augment humans in data storytelling. However, there lacks a systematic review to understand data storytelling tools from the perspective of human-AI collaboration, which hinders researchers from reflecting on the existing collaborative tool designs that promote humans' and AI's advantages and mitigate their shortcomings. This paper investigated existing tools with a framework from two perspectives: the stages in the storytelling workflow where a tool serves, including analysis, planning, implementation, and communication, and the roles of humans and AI in each stage, such as creators, assistants, optimizers, and reviewers. Through our analysis, we recognize the common collaboration patterns in existing tools, summarize lessons learned from these patterns, and further illustrate research opportunities for human-AI collaboration in data storytelling.