mlsanityData to 7 Oct 2026

MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization

arXiv 2402.11453 ↗First posted 18 Feb 2024cs.CLcited by 98

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Scientific data visualization plays a crucial role in research by enabling the direct display of complex information and assisting researchers in identifying implicit patterns. Despite its importance, the use of Large Language Models (LLMs) for scientific data visualization remains rather unexplored. In this study, we introduce MatPlotAgent, an efficient model-agnostic LLM agent framework designed to automate scientific data visualization tasks. Leveraging the capabilities of both code LLMs and multi-modal LLMs, MatPlotAgent consists of three core modules: query understanding, code generation with iterative debugging, and a visual feedback mechanism for error correction. To address the lack of benchmarks in this field, we present MatPlotBench, a high-quality benchmark consisting of 100 human-verified test cases. Additionally, we introduce a scoring approach that utilizes GPT-4V for automatic evaluation. Experimental results demonstrate that MatPlotAgent can improve the performance of various LLMs, including both commercial and open-source models. Furthermore, the proposed evaluation method shows a strong correlation with human-annotated scores.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 53 references that are in the map. First posted on arXiv 18 Feb 2024.

Builds on

29 of 53 references.

References · 29

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  10. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  11. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  12. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
17 more
  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  2. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  5. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  6. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  7. 2023Teaching Large Language Models to Self-Debugcited by 963
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  9. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  10. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  11. 2023Llemma: An Open Language Model For Mathematicscited by 385
  12. 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
  13. 2023SantaCoder: don't reach for the stars!cited by 169
  14. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  15. 2023WebCPM: Interactive Web Search for Chinese Long-form Question Answeringcited by 93
  16. 2023Is GPT-4 a Good Data Analyst?cited by 69
  17. 2018Data2Vis: Automatic Generation of Data Visualizations Using Sequence to Sequence Recurrent Neural Networkscited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 43; 2026: 43.

98 papers.

Sort:

By area:

  1. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
  2. 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
  3. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 208
  4. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 118
  5. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
  6. 2024Current state of LLM Risks and AI Guardrailscited by 44
  7. 2025METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scalingcited by 33
  8. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  9. 2025ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editingcited by 28
  10. 2024PyBench: Evaluating LLM Agent on various real-world coding taskscited by 24
  11. 2024Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performancecited by 22
  12. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
38 more
  1. 2025Large Language Model-based Data Science Agent: A Surveycited by 22
  2. 2026PaperBanana: Automating Academic Illustration for AI Scientistscited by 22
  3. 2024Drawing Pandas: A Benchmark for LLMs in Generating Plotting Codecited by 21
  4. 2025TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understandingcited by 20
  5. 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Frameworkcited by 20
  6. 2025Effective Training Data Synthesis for Improving MLLM Chart Understandingcited by 20
  7. 2026AutoFigure: Generating and Refining Publication-Ready Scientific Illustrationscited by 20
  8. 2025PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedbackcited by 19
  9. 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generationcited by 18
  10. 2025ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learningcited by 15
  11. 2025CoDA: Agentic Systems for Collaborative Data Visualizationcited by 14
  12. 2026IGenBench: Benchmarking the Reliability of Text-to-Infographic Generationcited by 14
  13. 2026DSGym: A Holistic Framework for Evaluating and Training Data Science Agentscited by 14
  14. 2024A Survey on Human-Centric LLMscited by 13
  15. 2024PoTable: Towards Systematic Thinking via Plan-then-Execute Stage Reasoning on Tablescited by 13
  16. 2025AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomycited by 13
  17. 2026AutoFigure-Edit: Generating Editable Scientific Illustrationcited by 12
  18. 2025ParaView-MCP: An Autonomous Visualization Agent with Direct Tool Usecited by 11
  19. 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinementcited by 10
  20. 2026SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agentscited by 9
  21. 2025DatawiseAgent: A Notebook-Centric LLM Agent Framework for Adaptive and Robust Data Science Automationcited by 8
  22. 2025VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generationcited by 8
  23. 2025VizGenie: Toward Self-Refining, Domain-Aware Workflows for Next-Generation Scientific Visualizationcited by 8
  24. 2025Cognify: Supercharging Gen-AI Workflows With Hierarchical Autotuningcited by 7
  25. 2025JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligencecited by 7
  26. 2024C2: Scalable Auto-Feedback for LLM-based Chart Generationcited by 6
  27. 2025Automated Visualization Code Synthesis via Multi-Path Reasoning and Feedback-Driven Optimizationcited by 6
  28. 2025An Evaluation-Centric Paradigm for Scientific Visualization Agentscited by 6
  29. 2026SciFig: Towards Automating Editable Figure Generation for Scientific Paperscited by 6
  30. 2026Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learningcited by 6
  31. 2025From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualizationcited by 5
  32. 2025Manimator: Transforming Research Papers into Visual Explanationscited by 5
  33. 2025AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Sciencecited by 4
  34. 2025BIASINSPECTOR: Detecting Bias in Structured Data through LLM Agentscited by 4
  35. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  36. 2026Hunt Instead of Wait: Evaluating Deep Data Research on Large Language Modelscited by 4
  37. 2026LongDS-Bench: On the Failure of Long-Horizon Agentic Data Analysiscited by 4
  38. 2025OpusAnimation: Code-Based Dynamic Chart Generationcited by 3

50 most cited of 98 All 98 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Agentic Authoring of Interactive Multiview Visualizations in Genomics
  2. 2026SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualization
  3. 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizations
  4. 2025nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow
  5. 2025LLM-Based Data Science Agents: A Survey of Capabilities, Challenges, and Future Directions
  6. 2026Figures as Interfaces: Toward LLM-Native Artifacts for Scientific Discovery
  7. 2026DeepXiv-SDK: An Agentic Data Interface for Scientific Literature
  8. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents