# Hidden flaws behind expert-level accuracy of multimodal GPT-4 vision in medicine

arXiv: https://arxiv.org/abs/2401.08396 · HTML: https://mlsanity.com/paper/2401.08396/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health](https://mlsanity.com/paper/2306.10070/) (2023)
- [Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosis](https://mlsanity.com/paper/2310.09909/) (2023)
- [Matching Patients to Clinical Trials with Large Language Models](https://mlsanity.com/paper/2307.15051/) (2023)
- [Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V](https://mlsanity.com/paper/2310.19061/) (2023)
- [Multimodal Foundation Models Exploit Text to Make Medical Image Predictions](https://mlsanity.com/paper/2311.05591/) (2023)
- [PubMed and Beyond: Biomedical Literature Search in the Age of Artificial Intelligence](https://mlsanity.com/paper/2307.09683/) (2023)

## Built on by (most cited first, 30 of 31)

- [Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions](https://mlsanity.com/paper/2402.18060/) (2024)
- [MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports](https://mlsanity.com/paper/2505.11733/) (2025)
- [ClinicalBench: Can LLMs Beat Traditional ML Models in Clinical Prediction?](https://mlsanity.com/paper/2411.06469/) (2024)
- [Superhuman performance of a large language model on the reasoning tasks of a physician](https://mlsanity.com/paper/2412.10849/) (2024)
- [Automating Expert-Level Medical Reasoning Evaluation of Large Language Models](https://mlsanity.com/paper/2507.07988/) (2025)
- [Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine](https://mlsanity.com/paper/2406.02394/) (2024)
- [Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Images](https://mlsanity.com/paper/2508.00549/) (2025)
- [The Illusion of Readiness in Health AI](https://mlsanity.com/paper/2509.18234/) (2025)
- [From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations](https://mlsanity.com/paper/2509.16584/) (2025)
- [A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challenges](https://mlsanity.com/paper/2411.00024/) (2024)
- [Capabilities of GPT-5 across critical domains: Is it the next breakthrough?](https://mlsanity.com/paper/2508.19259/) (2025)
- [Process-Supervised Reward Models for Verifying Clinical Note Generation: A Scalable Approach Guided by Domain Expertise](https://mlsanity.com/paper/2412.12583/) (2024)
- [MultiViewDx: Evidence-Linked Multi-View Clinical Diagnosis](https://mlsanity.com/paper/2410.14948/) (2024)
- [Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models](https://mlsanity.com/paper/2507.09209/) (2025)
- [Medical thinking with multiple images](https://mlsanity.com/paper/2604.16506/) (2026)
- [Entry-level guide to the use of large language models for medical research](https://mlsanity.com/paper/2410.18856/) (2024)
- [GRAID: Enhancing Spatial Reasoning of VLMs Through High-Fidelity Data Generation](https://mlsanity.com/paper/2510.22118/) (2025)
- [CT-Bench: A Benchmark for Multimodal Lesion Understanding in Computed Tomography](https://mlsanity.com/paper/2602.14879/) (2026)
- [A Real-World Evaluation of LLM Medication Safety Reviews in NHS Primary Care](https://mlsanity.com/paper/2512.21127/) (2025)
- [MedArena: Comparing LLMs for Medicine-in-the-Wild Clinician Preferences](https://mlsanity.com/paper/2603.15677/) (2026)
- [SynthEHR-Eviction: Enhancing Eviction SDoH Detection with LLM-Augmented Synthetic EHR Data](https://mlsanity.com/paper/2507.07421/) (2025)
- [Beyond Diagnosis: Evaluating Multimodal LLMs for Pathology Localization in Chest Radiographs](https://mlsanity.com/paper/2509.18015/) (2025)
- [Constructive Apraxia: An Unexpected Limit of Instructible Vision-Language Models and Analog for Human Cognitive Disorders](https://mlsanity.com/paper/2410.03551/) (2024)
- [PCS: Perceived Confidence Scoring of Black Box LLMs with Metamorphic Relations](https://mlsanity.com/paper/2502.07186/) (2025)
- [Vision Language Models versus Machine Learning Models Performance on Polyp Detection and Classification in Colonoscopy Images](https://mlsanity.com/paper/2503.21840/) (2025)
- [CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency](https://mlsanity.com/paper/2512.17213/) (2025)
- [SleepVLM: A Rule-Grounded Vision-Language Model for Auditable Sleep Staging](https://mlsanity.com/paper/2603.26738/) (2026)
- [Captioning Daily Activity Images in Early Childhood Education: Benchmark and Algorithm](https://mlsanity.com/paper/2604.01941/) (2026)
- [Agentic systems for breast cancer treatment recommendations](https://mlsanity.com/paper/2607.12051/) (2026)
- [CardioBench: A Real-World Data Benchmark for Evaluating Large Language Models in Clinically Authentic Cardiovascular Care Scenarios](https://mlsanity.com/paper/2607.25186/) (2026)

## Abstract (arXiv)

Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V's rationales of image comprehension, recall of medical knowledge, and step-by-step multimodal reasoning when solving New England Journal of Medicine (NEJM) Image Challenges - an imaging quiz designed to test the knowledge and diagnostic capabilities of medical professionals. Evaluation results confirmed that GPT-4V performs comparatively to human physicians regarding multi-choice accuracy (81.6% vs. 77.8%). GPT-4V also performs well in cases where physicians incorrectly answer, with over 78% accuracy. However, we discovered that GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%). Regardless of GPT-4V's high accuracy in multi-choice questions, our findings emphasize the necessity for further in-depth evaluations of its rationales before integrating such multimodal AI models into clinical workflows.