mlsanityData to 14 Oct 2026

Retrieval-Augmented VLMs for Multimodal Melanoma Diagnosis

arXiv 2509.08338 ↗First posted 10 Sep 2025cs.CV cs.AI cs.LGcited by 0

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

Accurate and early diagnosis of malignant melanoma is critical for improving patient outcomes. While convolutional neural networks (CNNs) have shown promise in dermoscopic image analysis, they often neglect clinical metadata and require extensive preprocessing. Vision-language models (VLMs) offer a multimodal alternative but struggle to capture clinical specificity when trained on general-domain data. To address this, we propose a retrieval-augmented VLM framework that incorporates semantically similar patient cases into the diagnostic prompt. Our method enables informed predictions without fine-tuning and significantly improves classification accuracy and error correction over conventional baselines. These results demonstrate that retrieval-augmented prompting provides a robust strategy for clinical decision support.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 14 of its 23 references that are in the map. First posted on arXiv 10 Sep 2025.

Builds on

14 of 23 references.

References · 14

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  7. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  8. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  9. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  10. 2024The Faiss librarycited by 376
  11. 2019BCN20000: Dermoscopic Lesions in the Wildcited by 225
  12. 2018Learn to Combine Modalities in Multimodal Deep Learningcited by 29
2 more
  1. 2024Can LLMs' Tuning Methods Work in Medical Multimodal Domain?cited by 12
  2. 2018Diagnostic Accuracy of Content Based Dermatoscopic Image Retrieval with Deep Classification Featurescited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions
  2. 2026How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective
  3. 2025MM-Skin: Enhancing Dermatology Vision-Language Model with an Image-Text Dataset Derived from Textbooks
  4. 2025Grounding Multimodal Large Language Models with Quantitative Skin Attributes: A Retrieval Study
  5. 2025Breast Cancer VLMs: Clinically Practical Vision-Language Train-Inference Models
  6. 2024Knowledge-grounded Adaptation Strategy for Vision-language Models: Building Unique Case-set for Screening Mammograms for Residents Training
  7. 2026CNNs, Transformers, Hybrid, and Vision Language Models for Skin Cancer Detection
  8. 2025Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review