# Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code

arXiv: https://arxiv.org/abs/2412.02764 · HTML: https://mlsanity.com/paper/2412.02764/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation](https://mlsanity.com/paper/2211.11501/) (2022)
- [ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation](https://mlsanity.com/paper/2406.09961/) (2024)
- [MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization](https://mlsanity.com/paper/2402.11453/) (2024)
- [LLMs for Science: Usage for Code Generation and Data Analysis](https://mlsanity.com/paper/2311.16733/) (2023)
- [nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task](https://mlsanity.com/paper/2112.12926/) (2021)

## Built on by (most cited first, 23 of 23)

- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [PaperBanana: Automating Academic Illustration for AI Scientists](https://mlsanity.com/paper/2601.23265/) (2026)
- [TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understanding](https://mlsanity.com/paper/2502.19400/) (2025)
- [Exploring LLM Agents for Cleaning Tabular Machine Learning Datasets](https://mlsanity.com/paper/2503.06664/) (2025)
- [VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation](https://mlsanity.com/paper/2506.03930/) (2025)
- [Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generation](https://mlsanity.com/paper/2504.09479/) (2025)
- [SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents](https://mlsanity.com/paper/2603.29139/) (2026)
- [JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence](https://mlsanity.com/paper/2510.23538/) (2025)
- [VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction](https://mlsanity.com/paper/2602.13294/) (2026)
- [An Evaluation-Centric Paradigm for Scientific Visualization Agents](https://mlsanity.com/paper/2509.15160/) (2025)
- [Code Aesthetics with Agentic Reward Feedback](https://mlsanity.com/paper/2510.23272/) (2025)
- [VisCoder2: Building Multi-Language Visualization Coding Agents](https://mlsanity.com/paper/2510.23642/) (2025)
- [Manimator: Transforming Research Papers into Visual Explanations](https://mlsanity.com/paper/2507.14306/) (2025)
- [InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generation](https://mlsanity.com/paper/2510.09724/) (2025)
- [RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluation](https://mlsanity.com/paper/2603.25804/) (2026)
- [PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generation](https://mlsanity.com/paper/2607.01883/) (2026)
- [MLDebugging: Towards Benchmarking Code Debugging Across Multi-Library Scenarios](https://mlsanity.com/paper/2506.13824/) (2025)
- [OpusAnimation: Code-Based Dynamic Chart Generation](https://mlsanity.com/paper/2510.03341/) (2025)
- [Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization](https://mlsanity.com/paper/2601.04582/) (2026)
- [Executing as You Generate: Hiding Execution Latency in LLM Code Interpreters](https://mlsanity.com/paper/2604.00491/) (2026)
- [VegaChat: A Robust Framework for LLM-Based Chart Generation and Assessment](https://mlsanity.com/paper/2601.15385/) (2026)
- [SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualization](https://mlsanity.com/paper/2606.05525/) (2026)
- [PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis](https://mlsanity.com/paper/2610.20538/) (2026)

## Abstract (arXiv)

This paper introduces the human-curated PandasPlotBench dataset, designed to evaluate language models' effectiveness as assistants in visual data exploration. Our benchmark focuses on generating code for visualizing tabular data - such as a Pandas DataFrame - based on natural language instructions, complementing current evaluation tools and expanding their scope. The dataset includes 175 unique tasks. Our experiments assess several leading Large Language Models (LLMs) across three visualization libraries: Matplotlib, Seaborn, and Plotly. We show that the shortening of tasks has a minimal effect on plotting capabilities, allowing for the user interface that accommodates concise user input without sacrificing functionality or accuracy. Another of our findings reveals that while LLMs perform well with popular libraries like Matplotlib and Seaborn, challenges persist with Plotly, highlighting areas for improvement. We hope that the modular design of our benchmark will broaden the current studies on generating visualizations. Our dataset and benchmark code are available online: https://huggingface.co/datasets/JetBrains-Research/PandasPlotBench; https://github.com/JetBrains-Research/PandasPlotBench.