mlsanityData to 14 Oct 2026

An Evaluation-Centric Paradigm for Scientific Visualization Agents

arXiv 2509.15160 ↗First posted 18 Sep 2025cs.HC cs.CL cs.GRcited by 6

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Recent advances in multi-modal large language models (MLLMs) have enabled increasingly sophisticated autonomous visualization agents capable of translating user intentions into data visualizations. However, measuring progress and comparing different agents remains challenging, particularly in scientific visualization (SciVis), due to the absence of comprehensive, large-scale benchmarks for evaluating real-world capabilities. This position paper examines the various types of evaluation required for SciVis agents, outlines the associated challenges, provides a simple proof-of-concept evaluation example, and discusses how evaluation benchmarks can facilitate agent self-improvement. We advocate for a broader collaboration to develop a SciVis agentic evaluation benchmark that would not only assess existing capabilities but also drive innovation and stimulate future development in the field.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 40 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

32 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splatting16 Jul
  2. 2025Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems25 May
  3. 2025ParaView-MCP: An Autonomous Visualization Agent with Direct Tool Use11 May
  4. 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization18 Feb
  5. 2023AVA: Towards Autonomous Visualization Agents through Visual Perception-Driven Decision-Making7 Dec

Other references · 27

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  4. 2024A Survey on LLM-as-a-Judgecited by 1,423
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  8. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  9. 2024Automated Design of Agentic Systemscited by 407
  10. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  11. 2025Small Language Models are the Future of Agentic AIcited by 270
  12. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
15 more
  1. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 183
  2. 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 125
  3. 2024Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Taskscited by 106
  4. 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 58
  5. 2024VisEval: A Benchmark for Data Visualization in the Era of Large Language Modelscited by 45
  6. 2025Magentic-UI: Towards Human-in-the-loop Agentic Systemscited by 43
  7. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
  8. 2024Visualization Generation with Large Language Models: An Evaluationcited by 29
  9. 2024Drawing Pandas: A Benchmark for LLMs in Generating Plotting Codecited by 23
  10. 2024ChatVis: Automating Scientific Visualization with a Large Language Modelcited by 11
  11. 2025VeriWeb: Verifiable Long-Chain Web Benchmark for Agentic Information-Seekingcited by 10
  12. 2023Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text)cited by 6
  13. 2024The Visualization JUDGE : Can Multimodal Foundation Models Guide Visualization Design Through Visual Perception?cited by 3
  14. 2024The State of Reproducibility Stamps for Visualization Research Paperscited by 2
  15. 2025Report on Challenges of Practical Reproducibility for Systems and HPC Computer Sciencecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agentscited by 10
  2. 2026Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Taskscited by 3
  3. 2026Exploring LLM Agent Designs and Interaction Modalities for Scientific Visualizationcited by 2
  4. 2026HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualizationcited by 2
  5. 2025KEO: Knowledge Extraction on OMIn via Knowledge Graphs and RAG for Safety-Critical Aviation Maintenancecited by 1
  6. 2026SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualizationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Multi-Agent Discovery and Resource-Aware Autonomous Exploration of Scientific Datasets
  2. 2025PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback
  3. 2025CoDA: Agentic Systems for Collaborative Data Visualization
  4. 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizations
  5. 2026Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
  6. 2026OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software
  7. 2025VizGenie: Toward Self-Refining, Domain-Aware Workflows for Next-Generation Scientific Visualization
  8. 2026The Truth, the Whole Truth, and Nothing but the Truth: Automatic Visualization Evaluation from Reconstruction Quality