mlsanityData to 7 Oct 2026

Gemini Goes to Med School: Exploring the Capabilities of Multimodal Large Language Models on Medical Challenge Problems & Hallucinations

arXiv 2402.07023 ↗First posted 10 Feb 2024cs.CL cs.AI cs.CVcited by 26

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large language models have the potential to be valuable in the healthcare industry, but it's crucial to verify their safety and effectiveness through rigorous evaluation. For this purpose, we comprehensively evaluated both open-source LLMs and Google's new multimodal LLM called Gemini across Medical reasoning, hallucination detection, and Medical Visual Question Answering tasks. While Gemini showed competence, it lagged behind state-of-the-art models like MedPaLM 2 and GPT-4 in diagnostic accuracy. Additionally, Gemini achieved an accuracy of 61.45% on the medical VQA dataset, significantly lower than GPT-4V's score of 88%. Our analysis revealed that Gemini is highly susceptible to hallucinations, overconfidence, and knowledge gaps, which indicate risks if deployed uncritically. We also performed a detailed analysis by medical subject and test type, providing actionable feedback for developers and clinicians. To mitigate risks, we applied prompting strategies that improved performance. Additionally, we facilitated future research and development by releasing a Python module for medical LLM evaluation and establishing a dedicated leaderboard on Hugging Face for medical domain LLMs. Python module can be found at https://github.com/promptslab/RosettaEval

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 27 references that are in the map. First posted on arXiv 10 Feb 2024.

Builds on

14 of 27 references.

References · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  11. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  12. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
2 more
  1. 2022Can large language models reason about medical questions?cited by 194
  2. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156

Built on by

Cited by year

Citing papers per year: 2024: 14; 2025: 12.

26 papers.

Sort:

By area:

  1. 2024Advancing Multimodal Medical Capabilities of Geminicited by 88
  2. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
  3. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  4. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23
  5. 2024Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Surveycited by 20
  6. 2025ReXVQA: A Large-scale Visual Question Answering Benchmark for Generalist Chest X-ray Understandingcited by 20
  7. 2024Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational Interviewingcited by 17
  8. 2024Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcarecited by 15
  9. 2024From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practicecited by 8
  10. 2024Reliable and diverse evaluation of LLM medical knowledge masterycited by 6
  11. 2025An evaluation of DeepSeek Models in Biomedical Natural Language Processingcited by 6
  12. 2024NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Visioncited by 5
14 more
  1. 2024Faculty Perspectives on the Potential of RAG in Computer Science Higher Educationcited by 3
  2. 2025MARK: Memory Augmented Refinement of Knowledgecited by 3
  3. 2025Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settingscited by 3
  4. 2024Pareto-Optimized Open-Source LLMs for Healthcare via Context Retrievalcited by 2
  5. 2024Image First or Text First? Optimising the Sequencing of Modalities in Large Language Model Prompting and Reasoning Taskscited by 2
  6. 2025A Scoping Review of Natural Language Processing in Addressing Medically Inaccurate Information: Errors, Misinformation, and Hallucinationcited by 2
  7. 2025Tonguescape: Exploring Language Models Understanding of Vowel Articulationcited by 1
  8. 2024Generative Artificial Intelligence-Guided User Studies: An Application for Air Taxi Servicescited by 0
  9. 2024Generative Language Models Potential for Requirement Engineering Applications: Insights into Current Strengths and Limitationscited by 0
  10. 2025Medalyze: Lightweight Medical Report Summarization Application Using FLAN-T5-Largecited by 0
  11. 2025OR-VSKC: Resolving Visual-Semantic Knowledge Conflicts in Operating Rooms with Synthetic Data-Guided Alignmentcited by 0
  12. 2025Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answeringcited by 0
  13. 2025BUSTR: Descriptor-Aware Vision-Language Learning for Breast Ultrasound Report Generationcited by 0
  14. 2025Toward Automatic Safe Driving Instruction: A Large-Scale Vision Language Model Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning
  2. 2024MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language Models
  3. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Models
  4. 2024MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
  5. 2025Demo: Statistically Significant Results On Biases and Errors of LLMs Do Not Guarantee Generalizable Results
  6. 2024MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context
  7. 2024Towards Reliable Medical Question Answering: Techniques and Challenges in Mitigating Hallucinations in Language Models
  8. 2024HALO: Hallucination Analysis and Learning Optimization to Empower LLMs with Retrieval-Augmented Context for Guided Clinical Decision Making