DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Data-driven storytelling is a powerful method for conveying insights by combining narrative techniques with visualizations and text. These stories integrate visual aids, such as highlighted bars and lines in charts, along with textual annotations explaining insights. However, creating such stories requires a deep understanding of the data and meticulous narrative planning, often necessitating human intervention, which can be time-consuming and mentally taxing. While Large Language Models (LLMs) excel in various NLP tasks, their ability to generate coherent and comprehensive data stories remains underexplored. In this work, we introduce a novel task for data story generation and a benchmark containing 1,449 stories from diverse sources. To address the challenges of crafting coherent data stories, we propose a multiagent framework employing two LLM agents designed to replicate the human storytelling process: one for understanding and describing the data (Reflection), generating the outline, and narration, and another for verification at each intermediary step. While our agentic framework generally outperforms non-agentic counterparts in both model-based and human evaluations, the results also reveal unique challenges in data story generation.
Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 37 of its 71 references that are in the map. First posted on arXiv 9 Aug 2024.
Builds on
37 of 71 references.
References · 37
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
- 2024Self-Rewarding Language Modelscited by 654
- 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
- 2019PlotQA: Reasoning over Scientific Plotscited by 317
- 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
- 2023OpenAGI: When LLM Meets Domain Expertscited by 196
- 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
- 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
25 more
- 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
- 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
- 2018StoryGAN: A Sequential Conditional GAN for Story Visualizationcited by 137
- 2023VisText: A Benchmark for Semantically Rich Chart Captioningcited by 111
- 2024Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineeringcited by 100
- 2020Content Planning for Neural Story Generation with Aristotelian Rescoringcited by 92
- 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84
- 2024ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoningcited by 68
- 2022OpenCQA: Open-ended Question Answering with Chartscited by 60
- 2018No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytellingcited by 46
- 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checkingcited by 34
- 2020Calliope: Automatic Visual Data Story Generation from a Spreadsheetcited by 30
- 2023ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summariescited by 30
- 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
- 2021Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoningcited by 29
- 2023ChartCheck: Explainable Fact-Checking over Real-World Chart Imagescited by 25
- 2023DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articlescited by 24
- 2024Creating Suspenseful Stories: Iterative Planning with Large Language Modelscited by 22
- 2018Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generationcited by 21
- 2023Improving Pacing in Long-Form Story Planningcited by 21
- 2022Erato: Cooperative Data Story Editing via Fact Interpolationcited by 11
- 2017Hierarchically-Attentive RNN for Album Summarization and Storytellingcited by 10
- 2021FairyTailor: A Multimodal Generative Framework for Storytellingcited by 9
- 2021Ordered Attention for Coherent Visual Storytellingcited by 6
- 2019Task-Oriented Optimal Sequencing of Visualization Chartscited by 4
Built on by
Citing papers per year: 2024: 1; 2025: 12; 2026: 12.
25 papers.
Sort:
By area:
- 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 105
- 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
- 2025Large Language Model-based Data Science Agent: A Surveycited by 22
- 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
- 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Frameworkcited by 20
- 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizationscited by 7
- 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directionscited by 5
- 2025The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Modelscited by 5
- 2026Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Modelscited by 4
- 2026TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualizationcited by 3
- 2025ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMscited by 2
- 2025Infogen: Generating Complex Statistical Infographics from Documentscited by 2
13 more
- 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualizationcited by 2
- 2026EvidFuse: Writing-Time Evidence Learning for Consistent Text-Chart Data Reportingcited by 2
- 2025MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Modelcited by 1
- 2025KAHAN: Knowledge-Augmented Hierarchical Analysis and Narration for Financial Data Narrationcited by 1
- 2025Multi-Agent Multimodal Large Language Model Framework for Automated Interpretation of Fuel Efficiency Analytics in Public Transportationcited by 1
- 2026DATAREEL: Automated Data-Driven Video Story Generation with Animationscited by 1
- 2026Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognitioncited by 0
- 2026Data Journalist Agent: Transforming Data into Verifiable Multimodal Storiescited by 0
- 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigationcited by 0
- 2026ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generationcited by 0
- 2026AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysiscited by 0
- 2026Behavior-Grounded Semantic Enrichment for Financial Fraud Modeling and Reasoningcited by 0
- 2026Towards Reliable AI Data Scientists: Data Agents with Workflow Harnessescited by 0
- 2026Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses28 Sep 2026
- 2026Behavior-Grounded Semantic Enrichment for Financial Fraud Modeling and Reasoning28 Sep 2026
- 2026AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis18 Aug 2026
- 2026ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation4 Aug 2026
- 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigation12 Jun 2026
- 2026Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories9 Jun 2026
- 2026DATAREEL: Automated Data-Driven Video Story Generation with Animations28 Apr 2026
- 2026TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualization26 Mar 2026
- 2026Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Models2 Feb 2026
- 2026Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognition2 Feb 2026
- 2026EvidFuse: Writing-Time Evidence Learning for Consistent Text-Chart Data Reporting9 Jan 2026
- 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization8 Jan 2026
13 more
- 2025Multi-Agent Multimodal Large Language Model Framework for Automated Interpretation of Fuel Efficiency Analytics in Public Transportation17 Nov 2025
- 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?27 Oct 2025
- 2025KAHAN: Knowledge-Augmented Hierarchical Analysis and Narration for Financial Data Narration21 Sep 2025
- 2025The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models13 Aug 2025
- 2025Large Language Model-based Data Science Agent: A Survey2 Aug 2025
- 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizations30 Jul 2025
- 2025Infogen: Generating Complex Statistical Infographics from Documents26 Jul 2025
- 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Framework3 Jun 2025
- 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering7 Apr 2025
- 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook14 Mar 2025
- 2025ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMs4 Feb 2025
- 2025MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model2 Jan 2025
- 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions29 Sep 2024
- 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
- 2025Large Language Model-based Data Science Agent: A Surveycited by 22
- 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Frameworkcited by 20
- 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizationscited by 7
- 2026Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Modelscited by 4
- 2026TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualizationcited by 3
- 2026AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysiscited by 0
- 2026Towards Reliable AI Data Scientists: Data Agents with Workflow Harnessescited by 0
- 2026Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses28 Sep 2026
- 2026AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis18 Aug 2026
- 2026TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualization26 Mar 2026
- 2026Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Models2 Feb 2026
- 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?27 Oct 2025
- 2025Large Language Model-based Data Science Agent: A Survey2 Aug 2025
- 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizations30 Jul 2025
- 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Framework3 Jun 2025
- 2025The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Modelscited by 5
- 2025Multi-Agent Multimodal Large Language Model Framework for Automated Interpretation of Fuel Efficiency Analytics in Public Transportationcited by 1
- 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigationcited by 0
- 2026ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generationcited by 0
- 2026ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation4 Aug 2026
- 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigation12 Jun 2026
- 2025Multi-Agent Multimodal Large Language Model Framework for Automated Interpretation of Fuel Efficiency Analytics in Public Transportation17 Nov 2025
- 2025The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Models13 Aug 2025
- 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 105
- 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlookcited by 20
- 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directionscited by 5
- 2025ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMscited by 2
- 2025Infogen: Generating Complex Statistical Infographics from Documentscited by 2
- 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualizationcited by 2
- 2026EvidFuse: Writing-Time Evidence Learning for Consistent Text-Chart Data Reportingcited by 2
- 2025MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Modelcited by 1
- 2025KAHAN: Knowledge-Augmented Hierarchical Analysis and Narration for Financial Data Narrationcited by 1
- 2026DATAREEL: Automated Data-Driven Video Story Generation with Animationscited by 1
- 2026Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognitioncited by 0
- 2026Data Journalist Agent: Transforming Data into Verifiable Multimodal Storiescited by 0
- 2026Data Journalist Agent: Transforming Data into Verifiable Multimodal Stories9 Jun 2026
- 2026DATAREEL: Automated Data-Driven Video Story Generation with Animations28 Apr 2026
- 2026Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognition2 Feb 2026
- 2026EvidFuse: Writing-Time Evidence Learning for Consistent Text-Chart Data Reporting9 Jan 2026
- 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization8 Jan 2026
- 2025KAHAN: Knowledge-Augmented Hierarchical Analysis and Narration for Financial Data Narration21 Sep 2025
- 2025Infogen: Generating Complex Statistical Infographics from Documents26 Jul 2025
- 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering7 Apr 2025
- 2025How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook14 Mar 2025
- 2025ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMs4 Feb 2025
- 2025MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model2 Jan 2025
- 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directions29 Sep 2024
Similar, not cited
Close in wording, no citation either way
- 2025StoryWriter: A Multi-Agent Framework for Long Story Generation
- 2024From Data to Story: Towards Automatic Animated Data Video Creation with LLM-based Multi-Agent Systems
- 2024Story3D-Agent: Exploring 3D Storytelling Visualization with Large Language Models
- 2024From Words to Worlds: Transforming One-line Prompt into Immersive Multi-modal Digital Stories with Communicative LLM Agent
- 2026DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytelling
- 2026DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration
- 2025Multi-Agent Data Visualization and Narrative Generation
- 2025Guiding Generative Storytelling with Knowledge Graphs