Performance of large language models in the optical diagnosis of colorectal polyps
Not on the map. Why?
Abstract (arXiv)
Background and Study Aims: Accurate optical diagnosis of colorectal polyps guides resection strategy and surveillance, with multimodal large language models (MLLMs) showing potential for image-based diagnosis. We aimed to evaluate the diagnostic accuracy of MLLMs in classifying colorectal polyps and predicting histology. Methods: We conducted a retrospective diagnostic performance study using the PRIME dataset, a curated set of white light and narrow-band imaging (NBI) images. We evaluated Claude Opus 4, Google Gemini 2.5 Pro, GPT-o3, GPT-4o, and GPT-5. For Paris, Narrow-band Imaging Colorectal Endoscopic (NICE), and predicted histology, we calculated F1 scores, percent correct scores, and accuracy of each MLLM compared to expert responses for 132 cases. Cochran's Q and McNemar's Test were used to determine differences between predicted values of each MLLM. Results: The F1 scores among MLLMs were >0.9 for all models for neoplastic vs. non-neoplastic polyps. Gemini 2.5 Pro demonstrated the highest F1 scores for invasive vs. non-invasive polyps and low- vs. high-grade adenoma, at 0.560 and 0.492 respectively. Claude Opus 4 and GPT-5 had statistically significantly higher percent correct scores than other MLLMs at 41.7%, using Paris classification. Conclusions: Claude Opus 4 and Gemini 2.5 Pro showed the highest accuracy in differentiating polyp subtypes, performing closest to expert consensus. Sensitivity and specificity, however, did not meet ESGE standards, highlighting the need for prospective multicenter trials and the design of human-in-the-loop workflows before clinical deployment.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 5 of its 33 references that are in the map. First posted on arXiv 31 Jul 2026.
Builds on
5 of 33 references.
References · 5
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
- 2024In-context learning enables multimodal large language models to classify cancer pathology imagescited by 38
- 2025Effects of Prompt Length on Domain-specific Tasks for Large Language Modelscited by 10
- 2024Are Longer Prompts Always Better? Prompt Selection in Large Language Models for Recommendation Systemscited by 6
- 2025DeepSeek-R1 Outperforms Gemini 2.0 Pro, OpenAI o1, and o3-mini in Bilingual Complex Ophthalmology Reasoningcited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Vision Language Models versus Machine Learning Models Performance on Polyp Detection and Classification in Colonoscopy Images
- 2025LDP: Parameter-Efficient Fine-Tuning of Multimodal LLM for Medical Report Generation
- 2025Diagnostic Accuracy of Open-Source Vision-Language Models on Diverse Medical Imaging Tasks
- 2024Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models
- 2026GI-Bench: A Panoramic Benchmark Revealing the Knowledge-Experience Dissociation of Multimodal Large Language Models in Gastrointestinal Endoscopy Against Clinical Standards
- 2025A Novel Ophthalmic Benchmark for Evaluating Multimodal Large Language Models with Fundus Photographs and OCT Images
- 2024Frontiers in Intelligent Colonoscopy
- 2025How Good is my Histopathology Vision-Language Foundation Model? A Holistic Benchmark