Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis
On the map: LLM agents and tool use → Medical and scientific domain agents
Abstract (arXiv)
State-of-the-art large language models (LLMs) show high performance in general visual question answering. However, a fundamental limitation remains: current architectures lack the native 3D spatial reasoning required to directly analyze volumetric medical imaging, such as CT or MRI. Emerging agentic AI offers a new solution, eliminating the need for intrinsic 3D processing by enabling LLMs to orchestrate and leverage specialized external tools. Yet, the feasibility of such agentic frameworks in complex, multi-step radiological workflows remains underexplored. In this work, we present a training-free agentic pipeline for automated brain MRI analysis. Validating our methodology on several LLMs (GPT-5.4, Gemini 3.1 Pro, Claude Sonnet 4.6) with off-the-shelf domain-specific tools, our system autonomously executes complex end-to-end workflows, including preprocessing (skull stripping, registration), pathology segmentation (glioma, meningioma, metastases), and volumetric reasoning. We evaluate our framework across increasingly complex radiological tasks, from single-scan tumor and anatomy analysis to longitudinal response assessment requiring multi-timepoint comparisons. We analyze the impact of architectural design by comparing single-agent models against multi-agent "domain-expert" collaborations. To support rigorous evaluation of future agentic systems, we release a benchmark dataset of image-prompt-answer tuples derived from public data. Our results demonstrate that agentic AI can solve highly neuro-radiological image analysis tasks through tool use without the need for training or fine-tuning.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 23 of its 36 references that are in the map. First posted on arXiv 17 Apr 2026.
Builds on
23 of 36 references.
Closest ancestors · 2
Same direction, newest first
- 2025A co-evolving agentic AI system for medical imaging analysis24 Sep
- 2025Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions28 Mar
Other references · 21
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
- 2021The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classificationcited by 507
- 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
- 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 235
- 2025Survey on Evaluation of LLM-based Agentscited by 190
- 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 154
- 2022SynthStrip: Skull-Stripping for Any Brain Imagecited by 126
- 2025MedRAX: Medical Reasoning Agent for Chest X-raycited by 68
- 2024The 2024 Brain Tumor Segmentation (BraTS) Challenge: Glioma Segmentation on Post-treatment MRIcited by 65
- 2023The ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: Intracranial Meningiomacited by 51
- 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysiscited by 44
9 more
- 2021The University of California San Francisco Preoperative Diffuse Glioma MRI (UCSF-PDGM) Datasetcited by 42
- 2023The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRIcited by 35
- 2024E3D-GPT: Enhanced 3D Visual Foundation for Medical Vision-Language Modelcited by 19
- 2023Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation mapscited by 11
- 2023The University of California San Francisco Brain Metastases Stereotactic Radiosurgery (UCSF-BMSR) MRI Datasetcited by 9
- 2025BraTS orchestrator : Democratizing and Disseminating state-of-the-art brain tumor image analysiscited by 8
- 2025OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Taskscited by 7
- 2025Performance of GPT-5 in Brain Tumor MRI Reasoningcited by 5
- 2025A Comparison and Evaluation of Fine-tuned Convolutional Neural Networks to Large Language Models for Image Classification and Segmentation of Brain Tumors on MRIcited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaborationcited by 2
- 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 1
- 2026Volumetric Radiology AI in the Era of Multimodal Large Language Modelscited by 0
- 2026A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scanscited by 0
- 2026A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scans21 Aug 2026
- 2026Volumetric Radiology AI in the Era of Multimodal Large Language Models20 Aug 2026
- 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy13 Jul 2026
- 2026Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration10 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025mAIstro: an open-source multi-agentic system for automated end-to-end development of radiomics and deep learning models for medical imaging
- 2026Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space
- 2025Agentic Systems in Radiology: Design, Applications, Evaluation, and Challenges
- 2026NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives
- 2026Route, Retrieve, Reflect, Repair: Self-Improving Agentic Framework for Visual Detection and Linguistic Reasoning in Medical Imaging
- 2026Can Coding Agents Build Robust Baselines? A Skill-Based Approach for Automating the Medical Imaging Model-Development Pipeline
- 2024How Well Can Modern LLMs Act as Agent Cores in Radiology Environments?
- 2026Agentic Automation of BT-RADS Scoring: End-to-End Multi-Agent System for Standardized Brain Tumor Follow-up Assessment