# Data Formulator: AI-powered Concept-driven Visualization Authoring

arXiv: https://arxiv.org/abs/2309.10094 · HTML: https://mlsanity.com/paper/2309.10094/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 70 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 26 of 26)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [Synchromesh: Reliable code generation from pre-trained language models](https://mlsanity.com/paper/2201.11227/) (2022)
- [LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models](https://mlsanity.com/paper/2303.02927/) (2023)
- [NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries](https://mlsanity.com/paper/2008.10723/) (2020)
- [Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networks](https://mlsanity.com/paper/1804.03126/) (2018)
- [Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations](https://mlsanity.com/paper/2110.00680/) (2021)
- [Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations](https://mlsanity.com/paper/1909.00031/) (2019)
- [mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks](https://mlsanity.com/paper/2009.10643/) (2020)
- [Lux: Always-on Visualization Recommendations for Exploratory Dataframe Workflows](https://mlsanity.com/paper/2105.00121/) (2021)
- [Falx: Synthesis-Powered Visualization Authoring](https://mlsanity.com/paper/2102.01024/) (2021)
- [Type-Directed Synthesis of Visualizations from Natural Language Queries](https://mlsanity.com/paper/2209.01081/) (2022)
- [Critical Reflections on Visualization Authoring Systems](https://mlsanity.com/paper/1907.13568/) (2019)
- [Visualization by Example](https://mlsanity.com/paper/1911.09668/) (2019)
- [Lyra 2: Designing Interactive Visualizations by Demonstration](https://mlsanity.com/paper/2008.09576/) (2020)
- [Animated Vega-Lite: Unifying Animation with a Grammar of Interactive Graphics](https://mlsanity.com/paper/2208.03869/) (2022)
- [Untidy Data: The Unreasonable Effectiveness of Tables](https://mlsanity.com/paper/2106.15005/) (2021)
- [Deconstructing Categorization in Visualization Recommendation: A Taxonomy and Comparative Study](https://mlsanity.com/paper/2102.07070/) (2021)
- [Revealing the Semantics of Data Wrangling Scripts With COMANTICS](https://mlsanity.com/paper/2209.13995/) (2022)

## Built on by (most cited first, 30 of 32)

- [nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow](https://mlsanity.com/paper/2502.05036/) (2025)
- [Prompt4Vis: Prompting Large Language Models with Example Mining and Schema Filtering for Tabular Data Visualization](https://mlsanity.com/paper/2402.07909/) (2024)
- [Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems](https://mlsanity.com/paper/2505.19101/) (2025)
- [GSurf: Learning Signed Distance Fields from Splatting Opaque Gaussians for High-quality 3D Reconstruction](https://mlsanity.com/paper/2411.15723/) (2024)
- [Data Analysis in the Era of Generative AI](https://mlsanity.com/paper/2409.18475/) (2024)
- [Exploring Multimodal Prompt for Visualization Authoring with Large Language Models](https://mlsanity.com/paper/2504.13700/) (2025)
- [Multi-Agent Data Visualization and Narrative Generation](https://mlsanity.com/paper/2509.00481/) (2025)
- [Contextualized Data-Wrangling Code Generation in Computational Notebooks](https://mlsanity.com/paper/2409.13551/) (2024)
- [HINTs: Sensemaking on large collections of documents with Hypergraph visualization and INTelligent agents](https://mlsanity.com/paper/2403.02752/) (2024)
- [ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMs](https://mlsanity.com/paper/2502.02329/) (2025)
- [Toward Real-World Table Agents: Capabilities, Workflows, and Design Principles for LLM-based Table Intelligence](https://mlsanity.com/paper/2507.10281/) (2025)
- [MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model](https://mlsanity.com/paper/2501.01014/) (2025)
- [Challenges & Opportunities with LLM-Assisted Visualization Retargeting](https://mlsanity.com/paper/2507.01436/) (2025)
- [A Generative AI System for Biomedical Data Discovery with Grammar-Based Visualizations](https://mlsanity.com/paper/2509.16454/) (2025)
- [YAC: Bridging Natural Language and Interactive Visual Exploration with Generative AI for Biomedical Data Discovery](https://mlsanity.com/paper/2509.19182/) (2025)
- [Evaluating the Semantic Profiling Abilities of LLMs for Natural Language Utterances in Data Visualization](https://mlsanity.com/paper/2407.06129/) (2024)
- [Data Therapist: Eliciting Domain Knowledge from Subject Matter Experts Using Large Language Models](https://mlsanity.com/paper/2505.00455/) (2025)
- [From Static to Interactive: Authoring Interactive Visualizations via Natural Language](https://mlsanity.com/paper/2601.17736/) (2026)
- [Toward Reliable Scientific Visualization Pipeline Construction with Structure-Aware Retrieval-Augmented LLMs](https://mlsanity.com/paper/2603.16057/) (2026)
- [Context-Mediated Domain Adaptation in Multi-Agent Sensemaking Systems](https://mlsanity.com/paper/2603.24858/) (2026)
- [Show Me the Infographic I Imagine: Intent-Aware Infographic Retrieval for Authoring Support](https://mlsanity.com/paper/2604.07989/) (2026)
- [Disentangling Visual and Factual Correctness in LVLMs' Visualization Literacy](https://mlsanity.com/paper/2606.03142/) (2026)
- [MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction](https://mlsanity.com/paper/2607.19910/) (2026)
- [MUSE: An Interactive Meta-Agent for Understanding and Steering LLM-powered Data Science Systems](https://mlsanity.com/paper/2608.16181/) (2026)
- [AdaLens: Interactive Storyline for Monitoring and Steering Long-Running Agentic Data Analysis](https://mlsanity.com/paper/2608.17834/) (2026)
- [Exploring Agentic Approaches for Data Issue Detection and Repair in AI-Assisted Visualization](https://mlsanity.com/paper/2608.21602/) (2026)
- [DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?](https://mlsanity.com/paper/2608.26757/) (2026)
- [Oculi: A Conversational Agentic Platform for Automated Credit Risk Analysis](https://mlsanity.com/paper/2608.28944/) (2026)
- [AI Soccer Analyst: Stage-Aware and Verifiable Human-AI Collaboration for Soccer Data Analysis](https://mlsanity.com/paper/2609.11224/) (2026)
- [VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queries](https://mlsanity.com/paper/2609.15182/) (2026)

## Abstract (arXiv)

With most modern visualization tools, authors need to transform their data into tidy formats to create visualizations they want. Because this requires experience with programming or separate data processing tools, data transformation remains a barrier in visualization authoring. To address this challenge, we present a new visualization paradigm, concept binding, that separates high-level visualization intents and low-level data transformation steps, leveraging an AI agent. We realize this paradigm in Data Formulator, an interactive visualization authoring tool. With Data Formulator, authors first define data concepts they plan to visualize using natural languages or examples, and then bind them to visual channels. Data Formulator then dispatches its AI-agent to automatically transform the input data to surface these concepts and generate desired visualizations. When presenting the results (transformed table and output visualizations) from the AI agent, Data Formulator provides feedback to help authors inspect and understand them. A user study with 10 participants shows that participants could learn and use Data Formulator to create visualizations that involve challenging data transformations, and presents interesting future research directions.