mlsanityData to 7 Oct 2026

SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

arXiv 2603.29139 ↗First posted 31 Mar 2026cs.AI cs.GR cs.HCcited by 9

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Recent advances in large language models (LLMs) have enabled agentic systems to translate natural-language intent into executable scientific visualization (SciVis) tasks. Despite rapid progress, the community lacks a principled and reproducible benchmark for evaluating these emerging SciVis agents in realistic, multi-step analysis settings. We present SciVisAgentBench, a comprehensive and extensible benchmark for evaluating scientific data analysis and visualization agents. Our benchmark is grounded in a structured taxonomy spanning four dimensions: application domain, data type, complexity level, and visualization operation. It currently comprises 108 expert-crafted cases covering diverse SciVis scenarios. To enable reliable assessment, we introduce a multimodal outcome-centric evaluation pipeline that combines LLM-based judging with deterministic evaluators, including image-based metrics, code checkers, rule-based verifiers, and case-specific evaluators. We also conduct a validity study with 12 SciVis experts to examine the agreement between human and LLM judges. Using this framework, we evaluate representative SciVis agents and general-purpose coding agents to establish initial baselines and reveal capability gaps. SciVisAgentBench is designed as a living benchmark to support systematic comparison, diagnose failure modes, and drive progress in agentic SciVis. The benchmark is available at https://scivisagentbench.github.io/.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 54 of its 90 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

54 of 90 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualization26 Mar
  2. 2025CoDA: Agentic Systems for Collaborative Data Visualization3 Oct
  3. 2025An Evaluation-Centric Paradigm for Scientific Visualization Agents18 Sep
  4. 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizations30 Jul
  5. 2025VizGenie: Toward Self-Refining, Domain-Aware Workflows for Next-Generation Scientific Visualization18 Jul
  6. 2025NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splatting16 Jul
  7. 2025ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks29 May
  8. 2025Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems25 May
  9. 2025ParaView-MCP: An Autonomous Visualization Agent with Direct Tool Use11 May
  10. 2025nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow7 Feb
  11. 2025PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback3 Feb
  12. 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization18 Feb
1 more
  1. 2023AVA: Towards Autonomous Visualization Agents through Visual Perception-Driven Decision-Making7 Dec

Other references · 41

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  6. 2024A Survey on LLM-as-a-Judgecited by 1,384
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  8. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  9. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  11. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  12. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 212
29 more
  1. 2024AI Agents That Mattercited by 199
  2. 2025Survey on Evaluation of LLM-based Agentscited by 190
  3. 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 169
  4. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 158
  5. 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Serverscited by 101
  6. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 94
  7. 2023LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Modelscited by 80
  8. 2020NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queriescited by 73
  9. 2024Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templatescited by 71
  10. 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 57
  11. 2024VisEval: A Benchmark for Data Visualization in the Era of Large Language Modelscited by 43
  12. 2025Evaluating Scoring Bias in LLM-as-a-Judgecited by 42
  13. 2025MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Modelscited by 34
  14. 2022DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualizationcited by 32
  15. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
  16. 2024Drawing Pandas: A Benchmark for LLMs in Generating Plotting Codecited by 21
  17. 2025AI Agents: Evolution, Architecture, and Real-World Applicationscited by 19
  18. 2025iVR-GS: Inverse Volume Rendering for Explorable Visualization via Editable 3D Gaussian Splattingcited by 17
  19. 2024Are LLMs ready for Visualization?cited by 16
  20. 2025LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queriescited by 15
  21. 2023The Rational Agent Benchmark for Data Visualizationcited by 10
  22. 2025TexGS-VolVis: Expressive Scene Editing for Volume Visualization via Textured Gaussian Splattingcited by 9
  23. 2023VOICE: Visual Oracle for Interaction, Conversation, and Explanationcited by 6
  24. 2025InferA: A Smart Assistant for Cosmological Ensemble Datacited by 6
  25. 2026SVLAT: Scientific Visualization Literacy Assessment Testcited by 6
  26. 2025Evaluating LLMs for Visualization Generation and Understandingcited by 5
  27. 2023Flexible and Probabilistic Topology Tracking with Partial Optimal Transportcited by 4
  28. 2025IntuiTF: MLLM-Guided Transfer Function Optimization for Direct Volume Renderingcited by 4
  29. 2025Automatic Semantic Alignment of Flow Pattern Representations for Exploration with Large Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 9
  2. 2026Exploring LLM Agent Designs and Interaction Modalities for Scientific Visualizationcited by 2
  3. 2026Toward AI VIS Co-Scientists: A General and End-to-End Agent Harness for Solving Complex Data Visualization Taskscited by 2
  4. 2026SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agentscited by 2
  5. 2026SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualizationcited by 1
  6. 2026HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualizationcited by 1
  7. 2026SCIRIGOR:Evaluating Open-Ended Scientific Analysis Beyond Final Scorescited by 0
  8. 2026Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agentscited by 0
  9. 2026Do Self-Evolving Skills Generalize to Held-Out Tasks?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
  2. 2026Multi-Agent Discovery and Resource-Aware Autonomous Exploration of Scientific Datasets
  3. 2024BLADE: Benchmarking Language Model Agents for Data-Driven Science
  4. 2026OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software
  5. 2026Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy
  6. 2026Agentic Authoring of Interactive Multiview Visualizations in Genomics
  7. 2026SciForge: An AI-Native, Multimodal Workbench for Scientific Discovery
  8. 2026TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents