# GPT-4V(ision) Unsuitable for Clinical Care and Education: A Clinician-Evaluated Assessment

arXiv: https://arxiv.org/abs/2403.12046 · HTML: https://mlsanity.com/paper/2403.12046/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 6 references that are in the map.
- First posted on arXiv 14 Nov 2023.

## Builds on (most cited first, 1 of 1)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)

## Built on by (most cited first, 6 of 6)

- [Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervision](https://mlsanity.com/paper/2407.06189/) (2024)
- [Multimodal Large Language Models for Medicine: A Comprehensive Survey](https://mlsanity.com/paper/2504.21051/) (2025)
- [Gemini Pro Defeated by GPT-4V: Evidence from Education](https://mlsanity.com/paper/2401.08660/) (2023)
- [OLIVE: Object Level In-Context Visual Embeddings](https://mlsanity.com/paper/2406.00872/) (2024)
- [An Early Investigation into the Utility of Multimodal Large Language Models in Medical Imaging](https://mlsanity.com/paper/2406.00667/) (2024)
- [Large language models in medical time series analysis](https://mlsanity.com/paper/2609.22262/) (2026)

## Abstract (arXiv)

OpenAI's large multimodal model, GPT-4V(ision), was recently developed for general image interpretation. However, less is known about its capabilities with medical image interpretation and diagnosis. Board-certified physicians and senior residents assessed GPT-4V's proficiency across a range of medical conditions using imaging modalities such as CT scans, MRIs, ECGs, and clinical photographs. Although GPT-4V is able to identify and explain medical images, its diagnostic accuracy and clinical decision-making abilities are poor, posing risks to patient safety. Despite the potential that large language models may have in enhancing medical education and delivery, the current limitations of GPT-4V in interpreting medical images reinforces the importance of appropriate caution when using it for clinical decision-making.