VizGenie: Toward Self-Refining, Domain-Aware Workflows for Next-Generation Scientific Visualization
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
We present VizGenie, a self-improving, agentic framework that advances scientific visualization through large language model (LLM) by orchestrating of a collection of domain-specific and dynamically generated modules. Users initially access core functionalities--such as threshold-based filtering, slice extraction, and statistical analysis--through pre-existing tools. For tasks beyond this baseline, VizGenie autonomously employs LLMs to generate new visualization scripts (e.g., VTK Python code), expanding its capabilities on-demand. Each generated script undergoes automated backend validation and is seamlessly integrated upon successful testing, continuously enhancing the system's adaptability and robustness. A distinctive feature of VizGenie is its intuitive natural language interface, allowing users to issue high-level feature-based queries (e.g., ``visualize the skull"). The system leverages image-based analysis and visual question answering (VQA) via fine-tuned vision models to interpret these queries precisely, bridging domain expertise and technical implementation. Additionally, users can interactively query generated visualizations through VQA, facilitating deeper exploration. Reliability and reproducibility are further strengthened by Retrieval-Augmented Generation (RAG), providing context-driven responses while maintaining comprehensive provenance records. Evaluations on complex volumetric datasets demonstrate significant reductions in cognitive overhead for iterative visualization tasks. By integrating curated domain-specific tools with LLM-driven flexibility, VizGenie not only accelerates insight generation but also establishes a sustainable, continuously evolving visualization practice. The resulting platform dynamically learns from user interactions, consistently enhancing support for feature-centric exploration and reproducible research in scientific visualization.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 46 references that are in the map. First posted on arXiv 18 Jul 2025.
Builds on
15 of 46 references.
Closest ancestors · 2
Same direction, newest first
- 2024LightVA: Lightweight Visual Analytics with LLM Agent-Based Task Planning and Execution8 Nov
- 2024MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization18 Feb
Other references · 13
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Visual Instruction Tuningcited by 7,869
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2021Towards Natural Language Interfaces for Data Visualization: A Surveycited by 59
- 2023Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Modelscited by 59
- 2024HAIChart: Human and AI Paired Visualization Systemcited by 26
- 2023LM4HPC: Towards Effective Language Model Application in High-Performance Computingcited by 18
- 2024FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learningcited by 16
- 2024ChatVis: Automating Scientific Visualization with a Large Language Modelcited by 11
- 2024PhenoFlow: A Human-LLM Driven Visual Analytics System for Exploring Large and Complex Stroke Datasetscited by 9
1 more
Built on by
Citing papers per year: 2025: 1; 2026: 7.
8 papers.
- 2026SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agentscited by 10
- 2025InferA: A Smart Assistant for Cosmological Ensemble Datacited by 6
- 2026TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualizationcited by 3
- 2026Exploring LLM Agent Designs and Interaction Modalities for Scientific Visualizationcited by 2
- 2026HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualizationcited by 2
- 2026SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualizationcited by 1
- 2026Toward Reliable Scientific Visualization Pipeline Construction with Structure-Aware Retrieval-Augmented LLMscited by 0
- 2026MedVA: An End-to-End Neuro-Symbolic Agentic System for Medical Volume Visualizationcited by 0
- 2026MedVA: An End-to-End Neuro-Symbolic Agentic System for Medical Volume Visualization14 Sep 2026
- 2026HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization25 Jun 2026
- 2026SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualization4 Jun 2026
- 2026Exploring LLM Agent Designs and Interaction Modalities for Scientific Visualization30 Apr 2026
- 2026SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents31 Mar 2026
- 2026TopoPilot: Reliable Conversational Workflow Automation for Topological Data Analysis and Visualization26 Mar 2026
- 2026Toward Reliable Scientific Visualization Pipeline Construction with Structure-Aware Retrieval-Augmented LLMs17 Mar 2026
- 2025InferA: A Smart Assistant for Cosmological Ensemble Data14 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2025PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback
- 2026Agentic Authoring of Interactive Multiview Visualizations in Genomics
- 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizations
- 2025VizGen: Data Exploration and Visualization from Natural Language via a Multi-Agent AI Architecture
- 2026Figures as Interfaces: Toward LLM-Native Artifacts for Scientific Discovery
- 2026InsightChain: Optimized Chain-of-Insight Analytics for LLM-driven Data Visualization
- 2025nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow
- 2026Crystalis: Progressive Nucleation and Semantic Annealing for Coordinated Multi-View Visualization Generation