mlsanityData to 14 Oct 2026

CT-Agent: A Multimodal-LLM Agent for 3D CT Radiology Question Answering

arXiv 2505.16229 ↗First posted 22 May 2025cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

Computed Tomography (CT) scan, which produces 3D volumetric medical data that can be viewed as hundreds of cross-sectional images (a.k.a. slices), provides detailed anatomical information for diagnosis. For radiologists, creating CT radiology reports is time-consuming and error-prone. A visual question answering (VQA) system that can answer radiologists' questions about some anatomical regions on the CT scan and even automatically generate a radiology report is urgently needed. However, existing VQA systems cannot adequately handle the CT radiology question answering (CTQA) task for: (1) anatomic complexity makes CT images difficult to understand; (2) spatial relationship across hundreds slices is difficult to capture. To address these issues, this paper proposes CT-Agent, a multimodal agentic framework for CTQA. CT-Agent adopts anatomically independent tools to break down the anatomic complexity; furthermore, it efficiently captures the across-slice spatial relationship with a global-local token compression strategy. Experimental results on two 3D chest CT datasets, CT-RATE and RadGenome-ChestCT, verify the superior performance of CT-Agent.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

25 of 39 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  3. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
  4. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  5. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  6. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  7. 2020MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysiscited by 206
  8. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 199
  9. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 162
  10. 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 114
  11. 2024CT2Rep: Automated Radiology Report Generation for 3D Medical Imagingcited by 96
  12. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 85
13 more
  1. 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 74
  2. 2024A Survey on LoRA of Large Language Modelscited by 67
  3. 2024RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysiscited by 42
  4. 2025PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathologycited by 39
  5. 2023MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Visioncited by 32
  6. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imagingcited by 22
  7. 2024Read Like a Radiologist: Efficient Vision-Language Model for 3D Medical Imaging Interpretationcited by 18
  8. 20243D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Modelscited by 16
  9. 2024CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT Volumescited by 10
  10. 2025RadSAM: Segmenting 3D radiological images with a 2D promptable modelcited by 6
  11. 2024Enhancing Single-Slice Segmentation with 3D-to-2D Unpaired Scan Distillationcited by 2
  12. 2024ViT3D Alignment of LLaMA3: 3D Medical Image Report Generationcited by 2
  13. 2025Explaining 3D Computed Tomography Classifiers with Counterfactualscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 17.

21 papers.

Sort:

By area:

  1. 2025Multi-step retrieval and reasoning improves radiology question answering with large language modelscited by 7
  2. 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretationcited by 7
  3. 2026RadAgent: A tool-using AI agent for stepwise interpretation of chest computed tomographycited by 6
  4. 2026MedOpenClaw and MedFlowBench: Auditing Medical Agents in Full-Study Workflowscited by 5
  5. 2025LungNoduleAgent: A Collaborative Multi-Agent System for Precision Diagnosis of Lung Nodulescited by 4
  6. 2026Beyond the Embedding Bottleneck: Adaptive Retrieval-Augmented 3D CT Report Generationcited by 3
  7. 2026DeepTumorVQA: A Hierarchical 3D CT Benchmark for Stage-Wise Evaluation of Medical VLMs and Tool-Augmented Agentscited by 3
  8. 2025Radiologist Copilot: An Agentic Framework Orchestrating Specialized Tools for Reliable Radiology Reportingcited by 2
  9. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 2
  10. 2026Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agentscited by 1
  11. 2026CT-Flow: Orchestrating CT Interpretation Workflow with Model Context Protocol Serverscited by 1
  12. 2026RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Reviewcited by 1
9 more
  1. 2026DiffVP: Differential Visual Semantic Prompting for LLM-Based CT Report Generationcited by 1
  2. 2026GAZE: Grounded Agentic Zero-shot Evaluation with Viewer-Level Tools and Literature Retrieval on Rare Brain MRIcited by 1
  3. 2026Policy-Driven CT-Agent: Modeling Phase-Aware Diagnostic Control for Clinically Consistent CT Reasoningcited by 1
  4. 2026Resolution Meets Reduction: Efficient Visual Context for 3D Radiology Report Generationcited by 1
  5. 2026MKG-CARE: Case-Aware Reasoning with Multimodal Knowledge Graphs for Explainable Medical Image Diagnosiscited by 0
  6. 2026Towards Reliable Fetal Ultrasound Interpretation with Multi-Agent Collaborationcited by 0
  7. 2026Volumetric Radiology AI in the Era of Multimodal Large Language Modelscited by 0
  8. 2026NV-Reason-CT: 3D Visual Language Model for CT Analysiscited by 0
  9. 2026ReVision3D: Attribution-Guided Recursive Self-Improvement for 3D Medical Perceptioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Computed Tomography Visual Question Answering with Cross-modal Feature Graphing
  2. 2025A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answering
  3. 2025RadImageNet-VQA: A Large-Scale CT and MRI Dataset for Radiologic Visual Question Answering
  4. 20253D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks
  5. 2025ReXVQA: A Large-scale Visual Question Answering Benchmark for Generalist Chest X-ray Understanding
  6. 2026Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging
  7. 2026Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space
  8. 2025MedErr-CT: A Visual Question Answering Benchmark for Identifying and Correcting Errors in CT Reports