mlsanityData to 7 Oct 2026

A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answering

arXiv 2508.02841 ↗First posted 4 Aug 2025cs.AI cs.IRcited by 1

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

Radiology visual question answering (RVQA) provides precise answers to questions about chest X-ray images, alleviating radiologists' workload. While recent methods based on multimodal large language models (MLLMs) and retrieval-augmented generation (RAG) have shown promising progress in RVQA, they still face challenges in factual accuracy, hallucinations, and cross-modal misalignment. We introduce a multi-agent system (MAS) designed to support complex reasoning in RVQA, with specialized agents for context understanding, multimodal reasoning, and answer validation. We evaluate our system on a challenging RVQA set curated via model disagreement filtering, comprising consistently hard cases across multiple MLLMs. Extensive experiments demonstrate the superiority and effectiveness of our system over strong MLLM baselines, with a case study illustrating its reliability and interpretability. This work highlights the potential of multi-agent approaches to support explainable and trustworthy clinical AI applications that require complex reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 67 of its 87 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

67 of 87 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow21 Mar
  2. 2025MedicalAgentsBench for Complex Medical Reasoning: Comparing Internalized Reasoning Models versus Externalized Agent-based Frameworks10 Mar
  3. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning16 Nov

Other references · 64

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2025Qwen2.5-VL Technical Reportcited by 5,418
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  9. 2017Billion-scale similarity search with GPUscited by 2,477
  10. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  12. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
38 more
  1. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  3. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  4. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  5. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  6. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  7. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  8. 2024Phi-4 Technical Reportcited by 675
  9. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  10. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  11. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  12. 2025MedGemma Technical Reportcited by 398
  13. 2025Accelerating scientific discovery with Co-Scientistcited by 353
  14. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  15. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  16. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 171
  17. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
  18. 2021What Makes Multi-modal Learning Better than Single (Provably)cited by 151
  19. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  20. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
  21. 2024Towards Building Multilingual Language Model for Medicinecited by 91
  22. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  23. 2024ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLMcited by 73
  24. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 66
  25. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  26. 2023XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Modelscited by 61
  27. 2024RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Modelscited by 58
  28. 2023EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Imagescited by 53
  29. 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoderscited by 49
  30. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  31. 2024MEDCO: Medical Education Copilots Based on A Multi-Agent Frameworkcited by 33
  32. 2023RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-trainingcited by 31
  33. 2023Masked Vision and Language Pre-training with Unimodal and Multimodal Contrastive Losses for Medical Visual Question Answeringcited by 29
  34. 2023Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Modelscited by 28
  35. 2024Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentationcited by 27
  36. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoningcited by 23
  37. 2024A Survey on Large Language Model-based Agents for Statistics and Data Sciencecited by 22
  38. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imagingcited by 21

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025CT-Agent: A Multimodal-LLM Agent for 3D CT Radiology Question Answering
  2. 2025Multi-step retrieval and reasoning improves radiology question answering with large language models
  3. 2025A Multimodal Multi-Agent Framework for Radiology Report Generation
  4. 2026Agentic retrieval-augmented reasoning reshapes collective reliability under model variability in radiology question answering
  5. 2025RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflows
  6. 2025Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation
  7. 2025Adapting Lightweight Vision Language Models for Radiological Visual Question Answering
  8. 2025Rethinking Information Synthesis in Multimodal Question Answering A Multi-Agent Perspective