mlsanityData to 14 Oct 2026

MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration

arXiv 2506.19835 ↗First posted 24 Jun 2025cs.CLcited by 40

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

Recent advancements in medical Large Language Models (LLMs) have showcased their powerful reasoning and diagnostic capabilities. Despite their success, current unified multimodal medical LLMs face limitations in knowledge update costs, comprehensiveness, and flexibility. To address these challenges, we introduce the Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis (MAM). Inspired by our empirical findings highlighting the benefits of role assignment and diagnostic discernment in LLMs, MAM decomposes the medical diagnostic process into specialized roles: a General Practitioner, Specialist Team, Radiologist, Medical Assistant, and Director, each embodied by an LLM-based agent. This modular and collaborative framework enables efficient knowledge updates and leverages existing medical LLMs and knowledge bases. Extensive experimental evaluations conducted on a wide range of publicly accessible multimodal medical datasets, incorporating text, image, audio, and video modalities, demonstrate that MAM consistently surpasses the performance of modality-specific LLMs. Notably, MAM achieves significant performance improvements ranging from 18% to 365% compared to baseline models. Our code is released at https://github.com/yczhou001/MAM.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 24 of its 46 references that are in the map. First posted on arXiv 24 Jun 2025.

Builds on

24 of 46 references.

References · 24

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  5. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  9. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  10. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  11. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  12. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
12 more
  1. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  2. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
  3. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  4. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 214
  5. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 200
  6. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 156
  7. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 134
  8. 2023DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultationcited by 90
  9. 2023MathChat: Converse to Tackle Challenging Math Problems with LLM Agentscited by 68
  10. 2023Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcarecited by 62
  11. 2017DeepLesion: Automated Deep Mining, Categorization and Detection of Significant Radiology Image Findings using Large-Scale Clinical Lesion Annotationscited by 24
  12. 2022Sound-Dr: Reliable Sound Dataset and Baseline Artificial Intelligence System for Respiratory Illnessescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 19.

40 papers.

Sort:

By area:

  1. 2025CXRAgent: Director-Orchestrated Multi-Stage Reasoning for Chest X-Ray Interpretationcited by 7
  2. 2025TheraMind: A Strategic and Adaptive Agent for Longitudinal Psychological Counselingcited by 7
  3. 2025ClinTutor-R1: Advancing Scalable and Robust One-to-Many Alignment in Clinical Socratic Educationcited by 6
  4. 2026Graph of States: Solving Abductive Tasks with Large Language Modelscited by 5
  5. 2026XrayClaw: Cooperative-Competitive Multi-Agent Alignment for Trustworthy Chest X-ray Diagnosiscited by 5
  6. 2025Coherent Multimodal Reasoning with Iterative Self-Evaluation for Vision-Language Modelscited by 3
  7. 2025ContextualLVLM-Agent: A Holistic Framework for Multi-Turn Visually-Grounded Dialogue and Complex Instruction Followingcited by 3
  8. 2025MAS-Shield: A Defense Framework for Secure and Efficient LLM MAScited by 3
  9. 2026Demystifying Low-Rank Knowledge Distillation in Large Language Models: Convergence, Generalization, and Information-Theoretic Guaranteescited by 3
  10. 2025MV-CoRe: Multimodal Visual-Conceptual Reasoning for Complex Visual Question Answeringcited by 2
  11. 2025Sim4Seg: Boosting Multimodal Multi-disease Medical Diagnosis Segmentation with Region-Aware Vision-Language Similarity Maskscited by 2
  12. 2026Do Mixed-Vendor Multi-Agent LLMs Improve Clinical Diagnosis?cited by 2
28 more
  1. 2026PVminerLLM: Structured Extraction of Patient Voice from Patient-Generated Text using Large Language Modelscited by 2
  2. 2026MetaAgent-X : Breaking the Ceiling of Automatic Multi-Agent Systems via End-to-End Reinforcement Learningcited by 2
  3. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 2
  4. 2025Large Language Models for Zero-Shot Multicultural Name Recognitioncited by 1
  5. 2025CIMR: Contextualized Iterative Multimodal Reasoning for Robust Instruction Following in LVLMscited by 1
  6. 2025CAP-LLM: Context-Augmented Personalized Large Language Models for News Headline Generationcited by 1
  7. 2025LumiGen: An LVLM-Enhanced Iterative Framework for Fine-Grained Text-to-Image Generationcited by 1
  8. 2025DeCoT: Decomposing Complex Instructions for Enhanced Text-to-Image Generation with Large Language Modelscited by 1
  9. 2025M3PO: Multimodal-Model-Guided Preference Optimization for Visual Instruction Followingcited by 1
  10. 2025XDR-LVLM: An Explainable Vision-Language Large Model for Diabetic Retinopathy Diagnosiscited by 1
  11. 2025Hierarchical Contextual Grounding LVLM: Enhancing Fine-Grained Visual-Language Understanding with Robust Groundingcited by 1
  12. 2025Context-Adaptive Synthesis and Compression for Enhanced Retrieval-Augmented Generation in Complex Domainscited by 1
  13. 2026Thinking Like a Clinician: A Cognitive AI Agent for Clinical Diagnosis via Panoramic Profiling and Adversarial Debatecited by 1
  14. 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoningcited by 1
  15. 2025Unlocking Compositional Control: Self-Supervision for LVLM-Based Image Generationcited by 0
  16. 2025LVLM-Composer's Explicit Planning for Image Generationcited by 0
  17. 2025ContextGuard-LVLM: Enhancing News Veracity through Fine-grained Cross-modal Contextual Consistency Verificationcited by 0
  18. 2025VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosiscited by 0
  19. 2026CoMMa: Contribution-Aware Medical Multi-Agents for Decentralized Oncology Decision Supportcited by 0
  20. 2026AE-LLM: Adaptive Efficiency Optimization for Large Language Modelscited by 0
  21. 2026MedRoute: RL-Based Dynamic Specialist Routing in Multi-Agent Medical Diagnosiscited by 0
  22. 2026TSAssistant: A Human-in-the-Loop Agentic Framework for Automated Target Safety Assessmentcited by 0
  23. 2026PVminerLLM2: Improving Structured Extraction of Patient Voice via Preference Optimizationcited by 0
  24. 2026GPAgentBench-2K: Benchmarking Large Language Model Agents in Complex Clinical Action Spacecited by 0
  25. 2026MedProb: Probing Internal Representations of Vision-Language Models for Medical Question Answeringcited by 0
  26. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0
  27. 2026When Does External Guidance Help LLM Reasoning? A Bias-Variance Theory of Guidance-Augmented GRPOcited by 0
  28. 2026TerraAgent: Demonstration-Grounded Multi-Agent Reasoning for Long-Context Remote Sensing Visual Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLM
  2. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agent
  3. 2026MedMASLab: A Unified Orchestration Framework for Benchmarking Multimodal Medical Multi-Agent Systems
  4. 2026AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks
  5. 2025MDTeamGPT: A Self-Evolving LLM-based Multi-Agent Framework for Multi-Disciplinary Team Medical Consultation
  6. 2026A Structured Debate-Mixture-of-Agents Framework for Complex Clinical Diagnostic Decision Support
  7. 2025A Knowledge-driven Adaptive Collaboration of LLMs for Enhancing Medical Decision-making
  8. 2026Meissa: Multi-modal Medical Agentic Intelligence