mlsanityData to 7 Oct 2026

A Multimodal Multi-Agent Framework for Radiology Report Generation

arXiv 2505.09787 ↗First posted 14 May 2025cs.AIcited by 8

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Radiology report generation (RRG) aims to automatically produce diagnostic reports from medical images, with the potential to enhance clinical workflows and reduce radiologists' workload. While recent approaches leveraging multimodal large language models (MLLMs) and retrieval-augmented generation (RAG) have achieved strong results, they continue to face challenges such as factual inconsistency, hallucination, and cross-modal misalignment. We propose a multimodal multi-agent framework for RRG that aligns with the stepwise clinical reasoning workflow, where task-specific agents handle retrieval, draft generation, visual analysis, refinement, and synthesis. Experimental results demonstrate that our approach outperforms a strong baseline in both automatic metrics and LLM-based evaluations, producing more accurate, structured, and interpretable reports. This work highlights the potential of clinically aligned multi-agent frameworks to support explainable and trustworthy clinical AI applications.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 62 references that are in the map. First posted on arXiv 14 May 2025.

Builds on

43 of 62 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Enhancing LLMs for Impression Generation in Radiology Reports through a Multi-Agent System6 Dec
  2. 2024MEDCO: Medical Education Copilots Based on A Multi-Agent Framework22 Aug
  3. 2024Enhancing Diagnostic Accuracy through Multi-Agent Conversations: Using Large Language Models to Mitigate Cognitive Bias26 Jan

Other references · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  9. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  10. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  11. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  12. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
28 more
  1. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  2. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
  3. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
  4. 2023R2GenGPT: Radiology Report Generation with Frozen LLMscited by 111
  5. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  6. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  7. 2023MAIRA-1: A specialised large multimodal model for radiology report generationcited by 72
  8. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 66
  9. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  10. 2024RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Modelscited by 58
  11. 2024Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiologycited by 45
  12. 2023Radiology-Llama2: Best-in-Class Large Language Model for Radiologycited by 42
  13. 2023RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-trainingcited by 31
  14. 2023Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Modelscited by 28
  15. 2023Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT modelscited by 27
  16. 2024Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentationcited by 27
  17. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23
  18. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoningcited by 23
  19. 2023Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Reviewcited by 20
  20. 2024Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancementcited by 16
  21. 2024Improving Medical Multi-modal Contrastive Learning with Expert Annotationscited by 16
  22. 2024Memory-based Cross-modal Semantic Alignment Network for Radiology Report Generationcited by 11
  23. 2024SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Informationcited by 11
  24. 2024Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generationcited by 10
  25. 2024Towards Interpretable Radiology Report Generation via Concept Bottlenecks using a Multi-Agentic RAGcited by 9
  26. 2023The Importance of Robust Features in Mitigating Catastrophic Forgettingcited by 7
  27. 2023Effectively Fine-tune to Improve Large Multimodal Models for Radiology Report Generationcited by 3
  28. 2023Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledgecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Generative AI for Healthcare: Fundamentals, Challenges, and Perspectivescited by 3
  2. 2025MAS2: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systemscited by 1
  3. 2026Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generationcited by 1
  4. 2026CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generationcited by 1
  5. 2026Route, Retrieve, Reflect, Repair: Self-Improving Agentic Framework for Visual Detection and Linguistic Reasoning in Medical Imagingcited by 0
  6. 2026MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correctioncited by 0
  7. 2026Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentationcited by 0
  8. 2026A Modular Agent for Reliable and Auditable Spatial Relation Verification in CT Scanscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation
  2. 2026MARCH: Multi-Agent Radiology Clinical Hierarchy for CT Report Generation
  3. 2026MAC-RRG: Iterative Multi-Agent Collaboration for X-ray Radiology Report Generation
  4. 2026AgentsEval: Clinically Faithful Evaluation of Medical Imaging Reports via Multi-Agent Reasoning
  5. 2025Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation
  6. 2026EviAgent: Evidence-Driven Agent for Radiology Report Generation
  7. 2025CBM-RAG: Demonstrating Enhanced Interpretability in Radiology Report Generation with Multi-Agent RAG and Concept Bottleneck Models
  8. 2025MRGAgents: A Multi-Agent Framework for Improved Medical Report Generation with Med-LVLMs