# HAIChart: Human and AI Paired Visualization System

arXiv: https://arxiv.org/abs/2406.11033 · HTML: https://mlsanity.com/paper/2406.11033/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 70 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [The Dawn of Natural Language to SQL: Are We Fully Ready?](https://mlsanity.com/paper/2406.01265/) (2024)
- [Is GPT-4 a Good Data Analyst?](https://mlsanity.com/paper/2305.15038/) (2023)
- [Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networks](https://mlsanity.com/paper/1804.03126/) (2018)
- [VizML: A Machine Learning Approach to Visualization Recommendation](https://mlsanity.com/paper/1808.04819/) (2018)
- [M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search](https://mlsanity.com/paper/1802.04394/) (2018)
- [Calliope: Automatic Visual Data Story Generation from a Spreadsheet](https://mlsanity.com/paper/2010.09975/) (2020)
- [nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task](https://mlsanity.com/paper/2112.12926/) (2021)
- [Are Large Language Models Good Statisticians?](https://mlsanity.com/paper/2406.07815/) (2024)
- [LLM4Vis: Explainable Visualization Recommendation using ChatGPT](https://mlsanity.com/paper/2310.07652/) (2023)
- [Generative AI for Visualization: State of the Art and Future Directions](https://mlsanity.com/paper/2404.18144/) (2024)
- [How Good is ChatGPT in Giving Advice on Your Visualization Design?](https://mlsanity.com/paper/2310.09617/) (2023)
- [KG4Vis: A Knowledge Graph-Based Approach for Visualization Recommendation](https://mlsanity.com/paper/2107.12548/) (2021)
- [Table2Charts: Recommending Charts by Learning Shared Table Representations](https://mlsanity.com/paper/2008.11015/) (2020)
- [MultiVision: Designing Analytical Dashboards with Deep Learning Based Recommendation](https://mlsanity.com/paper/2107.07823/) (2021)
- [PI2: End-to-end Interactive Visualization Interface Generation from Queries](https://mlsanity.com/paper/2107.08203/) (2021)
- [DashBot: Insight-Driven Dashboard Generation Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2208.01232/) (2022)
- [Leveraging Foundation Models for Crafting Narrative Visualization: A Survey](https://mlsanity.com/paper/2401.14010/) (2024)
- [Beyond Interestingness: Semantic and Context-Aware Natural Language Query Recommendations for Visual Data Analysis](https://mlsanity.com/paper/2201.04868/) (2022)
- [Reinforcement Learning with a Disentangled Universal Value Function for Item Recommendation](https://mlsanity.com/paper/2104.02981/) (2021)
- [MobileVisFixer: Tailoring Web Visualizations for Mobile Phones Leveraging an Explainable Reinforcement Learning Framework](https://mlsanity.com/paper/2008.06678/) (2020)
- [Personalized Visualization Recommendation](https://mlsanity.com/paper/2102.06343/) (2021)
- [Demonstration of PI2: Interactive Visualization Interface Generation for SQL Analysis in Notebook](https://mlsanity.com/paper/2201.05664/) (2022)

## Built on by (most cited first, 26 of 26)

- [AFlow: Automating Agentic Workflow Generation](https://mlsanity.com/paper/2410.10762/) (2024)
- [A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?](https://mlsanity.com/paper/2408.05109/) (2024)
- [Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executors](https://mlsanity.com/paper/2504.04785/) (2025)
- [PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback](https://mlsanity.com/paper/2502.00988/) (2025)
- [IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation](https://mlsanity.com/paper/2601.04498/) (2026)
- [ParaView-MCP: An Autonomous Visualization Agent with Direct Tool Use](https://mlsanity.com/paper/2505.07064/) (2025)
- [DebFlow: Automating Agent Creation via Agent Debate](https://mlsanity.com/paper/2503.23781/) (2025)
- [VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing](https://mlsanity.com/paper/2605.15677/) (2026)
- [VizGenie: Toward Self-Refining, Domain-Aware Workflows for Next-Generation Scientific Visualization](https://mlsanity.com/paper/2507.21124/) (2025)
- [DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoning](https://mlsanity.com/paper/2508.01700/) (2025)
- [Automated Visualization Code Synthesis via Multi-Path Reasoning and Feedback-Driven Optimization](https://mlsanity.com/paper/2502.11140/) (2025)
- [Graph Neural Networks for Databases: A Survey](https://mlsanity.com/paper/2502.12908/) (2025)
- [ChartMark: A Structured Grammar for Chart Annotation](https://mlsanity.com/paper/2507.21810/) (2025)
- [ChartCards: A Chart-Metadata Generation Framework for Multi-Task Chart Understanding](https://mlsanity.com/paper/2505.15046/) (2025)
- [Trade in Minutes! Rationality-Driven Agentic System for Quantitative Financial Trading](https://mlsanity.com/paper/2510.04787/) (2025)
- [MultiVis-Agent: A Multi-Agent Framework with Logic Rules for Reliable and Comprehensive Cross-Modal Data Visualization](https://mlsanity.com/paper/2601.18320/) (2026)
- [ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMs](https://mlsanity.com/paper/2502.02329/) (2025)
- [Perovskite-LLM: Knowledge-Enhanced Large Language Models for Perovskite Solar Cell Research](https://mlsanity.com/paper/2502.12669/) (2025)
- [BayesFlow: A Probability Inference Framework for Meta-Agent Assisted Workflow Generation](https://mlsanity.com/paper/2601.22305/) (2026)
- [ViviDoc: Generating Interactive Documents through Human-Agent Collaboration](https://mlsanity.com/paper/2603.27991/) (2026)
- [DataMagic: Transforming Tabular Data into Data Insight Video](https://mlsanity.com/paper/2606.20388/) (2026)
- [MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model](https://mlsanity.com/paper/2501.01014/) (2025)
- [(P)rior(D)yna(F)low: A Priori Dynamic Workflow Construction via Multi-Agent Collaboration](https://mlsanity.com/paper/2509.14547/) (2025)
- [AgentSlimming: Towards Efficient and Cost-Aware Multi-Agent Systems](https://mlsanity.com/paper/2605.08813/) (2026)
- [Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration](https://mlsanity.com/paper/2603.01912/) (2026)
- [DataVista: Diagnosing Multimodal LLMs on Data Video Understanding](https://mlsanity.com/paper/2610.11993/) (2026)

## Abstract (arXiv)

The growing importance of data visualization in business intelligence and data science emphasizes the need for tools that can efficiently generate meaningful visualizations from large datasets. Existing tools fall into two main categories: human-powered tools (e.g., Tableau and PowerBI), which require intensive expert involvement, and AI-powered automated tools (e.g., Draco and Table2Charts), which often fall short of guessing specific user needs. In this paper, we aim to achieve the best of both worlds. Our key idea is to initially auto-generate a set of high-quality visualizations to minimize manual effort, then refine this process iteratively with user feedback to more closely align with their needs. To this end, we present HAIChart, a reinforcement learning-based framework designed to iteratively recommend good visualizations for a given dataset by incorporating user feedback. Specifically, we propose a Monte Carlo Graph Search-based visualization generation algorithm paired with a composite reward function to efficiently explore the visualization space and automatically generate good visualizations. We devise a visualization hints mechanism to actively incorporate user feedback, thus progressively refining the visualization generation module. We further prove that the top-k visualization hints selection problem is NP-hard and design an efficient algorithm. We conduct both quantitative evaluations and user studies, showing that HAIChart significantly outperforms state-of-the-art human-powered tools (21% better at Recall and 1.8 times faster) and AI-powered automatic tools (25.1% and 14.9% better in terms of Hit@3 and R10@30, respectively).