mlsanityData to 14 Oct 2026

Collaboration among Multiple Large Language Models for Medical Question Answering

arXiv 2505.16648 ↗First posted 22 May 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Empowered by vast internal knowledge reservoir, the new generation of large language models (LLMs) demonstrate untapped potential to tackle medical tasks. However, there is insufficient effort made towards summoning up a synergic effect from multiple LLMs' expertise and background. In this study, we propose a multi-LLM collaboration framework tailored on a medical multiple-choice questions dataset. Through post-hoc analysis on 3 pre-trained LLM participants, our framework is proved to boost all LLMs reasoning ability as well as alleviate their divergence among questions. We also measure an LLM's confidence when it confronts with adversary opinions from other LLMs and observe a concurrence between LLM's confidence and prediction accuracy.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

18 of 30 references.

References · 18

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2024Mixtral of Expertscited by 1,823
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  10. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  11. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  12. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
6 more
  1. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  2. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 222
  3. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 180
  4. 2023Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encodingcited by 71
  5. 2024Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approachescited by 52
  6. 2024Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Assessing Large Language Models for Medical QA: Zero-Shot and LLM-as-a-Judge Evaluationcited by 2
  2. 2026MediHive: A Decentralized Agent Collective for Medical Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models
  2. 2025Structured Outputs Enable General-Purpose LLMs to be Medical Experts
  3. 2026Let LLMs Judge Each Other: Multi-Agent Peer-Reviewed Reasoning for Medical Question Answering
  4. 2025CURE: Confidence-driven Unified Reasoning Ensemble Framework for Medical Question Answering
  5. 2023Aligning Large Language Models for Clinical Tasks
  6. 2024A Survey for Large Language Models in Biomedicine
  7. 2024The Limited Impact of Medical Adaptation of Large Language and Vision-Language Models
  8. 2025It is Too Many Options: Pitfalls of Multiple-Choice Questions in Generative AI and Medical Education