# Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis arXiv: https://arxiv.org/abs/2604.16729 ยท HTML: https://mlsanity.com/paper/2604.16729/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. - Builds on 23 of its 36 references that are in the map. - First posted on arXiv 17 Apr 2026. ## Placed in - LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/) ## Builds on (most cited first, 23 of 23) - [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023) - [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023) - [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021) - [Med-Flamingo: a Multimodal Medical Few-shot Learner](https://mlsanity.com/paper/2307.15189/) (2023) - [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021) - [Survey on Evaluation of LLM-based Agents](https://mlsanity.com/paper/2503.16416/) (2025) - [M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models](https://mlsanity.com/paper/2404.00578/) (2024) - [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022) - [MedRAX: Medical Reasoning Agent for Chest X-ray](https://mlsanity.com/paper/2502.02673/) (2025) - [The 2024 Brain Tumor Segmentation (BraTS) Challenge: Glioma Segmentation on Post-treatment MRI](https://mlsanity.com/paper/2405.18368/) (2024) - [The ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: Intracranial Meningioma](https://mlsanity.com/paper/2305.07642/) (2023) - [Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis](https://mlsanity.com/paper/2503.20047/) (2025) - [The University of California San Francisco Preoperative Diffuse Glioma MRI (UCSF-PDGM) Dataset](https://mlsanity.com/paper/2109.00356/) (2021) - [Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions](https://mlsanity.com/paper/2503.22678/) (2025) - [The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI](https://mlsanity.com/paper/2306.00838/) (2023) - [E3D-GPT: Enhanced 3D Visual Foundation for Medical Vision-Language Model](https://mlsanity.com/paper/2410.14200/) (2024) - [A co-evolving agentic AI system for medical imaging analysis](https://mlsanity.com/paper/2509.20279/) (2025) - [Panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps](https://mlsanity.com/paper/2312.02608/) (2023) - [The University of California San Francisco Brain Metastases Stereotactic Radiosurgery (UCSF-BMSR) MRI Dataset](https://mlsanity.com/paper/2304.07248/) (2023) - [BraTS orchestrator : Democratizing and Disseminating state-of-the-art brain tumor image analysis](https://mlsanity.com/paper/2506.13807/) (2025) - [OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks](https://mlsanity.com/paper/2511.00846/) (2025) - [Performance of GPT-5 in Brain Tumor MRI Reasoning](https://mlsanity.com/paper/2508.10865/) (2025) - [A Comparison and Evaluation of Fine-tuned Convolutional Neural Networks to Large Language Models for Image Classification and Segmentation of Brain Tumors on MRI](https://mlsanity.com/paper/2509.10683/) (2025) ## Built on by (most cited first, 4 of 4) - [Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration](https://mlsanity.com/paper/2605.09366/) (2026) - [The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy](https://mlsanity.com/paper/2607.11175/) (2026) - [Volumetric Radiology AI in the Era of Multimodal Large Language Models](https://mlsanity.com/paper/2608.20549/) (2026) - [A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scans](https://mlsanity.com/paper/2608.21140/) (2026) ## Abstract (arXiv) State-of-the-art large language models (LLMs) show high performance in general visual question answering. However, a fundamental limitation remains: current architectures lack the native 3D spatial reasoning required to directly analyze volumetric medical imaging, such as CT or MRI. Emerging agentic AI offers a new solution, eliminating the need for intrinsic 3D processing by enabling LLMs to orchestrate and leverage specialized external tools. Yet, the feasibility of such agentic frameworks in complex, multi-step radiological workflows remains underexplored. In this work, we present a training-free agentic pipeline for automated brain MRI analysis. Validating our methodology on several LLMs (GPT-5.4, Gemini 3.1 Pro, Claude Sonnet 4.6) with off-the-shelf domain-specific tools, our system autonomously executes complex end-to-end workflows, including preprocessing (skull stripping, registration), pathology segmentation (glioma, meningioma, metastases), and volumetric reasoning. We evaluate our framework across increasingly complex radiological tasks, from single-scan tumor and anatomy analysis to longitudinal response assessment requiring multi-timepoint comparisons. We analyze the impact of architectural design by comparing single-agent models against multi-agent "domain-expert" collaborations. To support rigorous evaluation of future agentic systems, we release a benchmark dataset of image-prompt-answer tuples derived from public data. Our results demonstrate that agentic AI can solve highly neuro-radiological image analysis tasks through tool use without the need for training or fine-tuning.