mlsanityData to 7 Oct 2026

MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning

arXiv 2311.10537 ↗First posted 16 Nov 2023cs.CL cs.AIcited by 311

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Large language models (LLMs), despite their remarkable progress across various general domains, encounter significant barriers in medicine and healthcare. This field faces unique challenges such as domain-specific terminologies and reasoning over specialized knowledge. To address these issues, we propose MedAgents, a novel multi-disciplinary collaboration framework for the medical domain. MedAgents leverages LLM-based agents in a role-playing setting that participate in a collaborative multi-round discussion, thereby enhancing LLM proficiency and reasoning capabilities. This training-free framework encompasses five critical steps: gathering domain experts, proposing individual analyses, summarising these analyses into a report, iterating over discussions until a consensus is reached, and ultimately making a decision. Our work focuses on the zero-shot setting, which is applicable in real-world scenarios. Experimental results on nine datasets (MedQA, MedMCQA, PubMedQA, and six subtasks from MMLU) establish that our proposed MedAgents framework excels at mining and harnessing the medical expertise within LLMs, as well as extending its reasoning abilities. Our code can be found at https://github.com/gersteinlab/MedAgents.

Key facts. Cited by 311 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 54 of its 95 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

54 of 95 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Can large language models reason about medical questions?17 Jul

Other references · 53

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
38 more
  1. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  2. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  6. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  7. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  10. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  11. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  12. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  13. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  14. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  15. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  16. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  17. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  18. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  19. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  20. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  21. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
  22. 2023Towards Generalist Biomedical AIcited by 286
  23. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  24. 2023RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Modelscited by 173
  25. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  26. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  27. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 129
  28. 2023Hallucinations in Large Multilingual Translation Modelscited by 126
  29. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
  30. 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
  31. 2023GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Informationcited by 115
  32. 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicinecited by 99
  33. 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 88
  34. 2023MetaAgents: Large Language Model Based Agents for Decision-Making on Teamingcited by 85
  35. 2023InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuningcited by 80
  36. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
  37. 2022Legal Prompting: Teaching a Language Model to Think Like a Lawyercited by 60
  38. 2023Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaborationcited by 52

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 79; 2025: 117; 2026: 114.

311 papers.

Sort:

By area:

  1. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  3. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  4. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  5. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 215
  6. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 210
  7. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
  8. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  9. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  10. 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 182
  11. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attackscited by 118
  12. 2024Empowering Biomedical Discovery with AI Agentscited by 117
38 more
  1. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 108
  2. 2025Multi-Agent Collaboration via Evolving Orchestrationcited by 102
  3. 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulatorcited by 99
  4. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 97
  5. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 96
  6. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  7. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 84
  8. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 84
  9. 2024AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planningcited by 81
  10. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  11. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  12. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  13. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  14. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 75
  15. 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directionscited by 74
  16. 2025Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?cited by 74
  17. 2025MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflowcited by 71
  18. 2024Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Datacited by 65
  19. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  20. 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agentscited by 61
  21. 2024KG-RAG: Bridging the Gap Between Knowledge and Creativitycited by 55
  22. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  23. 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
  24. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 49
  25. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  26. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 46
  27. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  28. 2025MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systemscited by 45
  29. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  30. 2025Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactionscited by 40
  31. 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Modelscited by 39
  32. 2025MedicalAgentsBench for Complex Medical Reasoning: Comparing Internalized Reasoning Models versus Externalized Agent-based Frameworkscited by 38
  33. 2025MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoningcited by 37
  34. 2025KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systemscited by 36
  35. 2024MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Educationcited by 35
  36. 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agentscited by 35
  37. 2024AgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learningcited by 33
  38. 2024MEDCO: Medical Education Copilots Based on A Multi-Agent Frameworkcited by 33

50 most cited of 311 All 311 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey for Large Language Models in Biomedicine
  2. 2025MedBrowseComp: Benchmarking Medical Deep Research and Computer Use
  3. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  4. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
  5. 2025MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents
  6. 2025MedAgentBoard: Benchmarking Multi-Agent Collaboration with Conventional Methods for Diverse Medical Tasks
  7. 2025Knowledge-Driven Agentic Scientific Corpus Distillation Framework for Biomedical Large Language Models Training
  8. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review