mlsanityData to 7 Oct 2026

MM-Skin: Enhancing Dermatology Vision-Language Model with an Image-Text Dataset Derived from Textbooks

arXiv 2505.06152 ↗First posted 9 May 2025cs.CV cs.AIcited by 14

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Medical vision-language models (VLMs) have shown promise as clinical assistants across various medical fields. However, specialized dermatology VLM capable of delivering professional and detailed diagnostic analysis remains underdeveloped, primarily due to less specialized text descriptions in current dermatology multimodal datasets. To address this issue, we propose MM-Skin, the first large-scale multimodal dermatology dataset that encompasses 3 imaging modalities, including clinical, dermoscopic, and pathological and nearly 10k high-quality image-text pairs collected from professional textbooks. In addition, we generate over 27k diverse, instruction-following vision question answering (VQA) samples (9 times the size of current largest dermatology VQA dataset). Leveraging public datasets and MM-Skin, we developed SkinVL, a dermatology-specific VLM designed for precise and nuanced skin disease interpretation. Comprehensive benchmark evaluations of SkinVL on VQA, supervised fine-tuning (SFT) and zero-shot classification tasks across 8 datasets, reveal its exceptional performance for skin diseases in comparison to both general and medical VLM models. The introduction of MM-Skin and SkinVL offers a meaningful contribution to advancing the development of clinical dermatology VLM assistants. MM-Skin is available at https://github.com/ZwQ803/MM-Skin

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 50 references that are in the map. First posted on arXiv 9 May 2025.

Builds on

30 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI21 Nov
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day1 Jun
  3. 2021Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training24 May

Other references · 27

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  6. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  7. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  8. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  9. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  10. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  11. 2019BCN20000: Dermoscopic Lesions in the Wildcited by 224
  12. 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
15 more
  1. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  2. 2021Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Datasetcited by 184
  3. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 169
  4. 2021Disparities in Dermatology AI: Assessments Using Diverse Clinical Imagescited by 132
  5. 2020PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphonescited by 116
  6. 2022Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Trainingcited by 104
  7. 2019Overcoming Data Limitation in Medical Visual Question Answeringcited by 71
  8. 2023SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysiscited by 44
  9. 2024RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysiscited by 41
  10. 2024MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosiscited by 28
  11. 2018The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesionscited by 26
  12. 2024MM-Retinal: Knowledge-Enhanced Foundational Pretraining with Fundus Image-Text Expertisecited by 24
  13. 2024Crowdsourcing Dermatology Images with Google Search Ads: Creating a Real-World Skin Condition Datasetcited by 21
  14. 2022RepsNet: Combining Vision with Language for Automated Medical Reportscited by 14
  15. 2023EasyGen: Easing Multimodal Generation with BiDiffuser and LLMscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025From Learning to Unlearning: Biomedical Security Protection in Multimodal Large Language Modelscited by 3
  2. 2026DermoGPT: Open Weights and Open Data for Morphology-Grounded Dermatological Reasoning MLLMscited by 3
  3. 2026SkinCLIP-VL: Consistency-Aware Vision-Language Learning for Multimodal Skin Cancer Diagnosiscited by 3
  4. 2025See-in-Pairs: Reference Image-Guided Comparative Vision-Language Models for Medical Diagnosiscited by 2
  5. 2025Towards Trustworthy Dermatology MLLMs: A Benchmark and Multimodal Evaluator for Diagnostic Narrativescited by 2
  6. 2025Anatomical Region-Guided Contrastive Decoding: A Plug-and-Play Strategy for Mitigating Hallucinations in Medical VLMscited by 2
  7. 2025Grounding Multimodal Large Language Models with Quantitative Skin Attributes: A Retrieval Studycited by 0
  8. 2025DermAI: Clinical dermatology acquisition through quality-driven image collection for AI classification in mobilecited by 0
  9. 2026A Hierarchical Benchmark of Foundation Models for Dermatologycited by 0
  10. 2026DerMAE: Improving skin lesion classification through conditioned latent diffusion and MAE distillationcited by 0
  11. 2026ReXInTheWild: A Unified Benchmark for Medical Photograph Understandingcited by 0
  12. 2026DermAgent: A Self-Reflective Agentic System for Dermatological Image Analysis with Multi-Tool Reasoning and Traceable Decision-Makingcited by 0
2 more
  1. 2026CoFiDA-M: Concept-Aware Feature Modulation for Cross-Domain Adaptation with Image-Only Inferencecited by 0
  2. 2026How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspectivecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DermaBench: A Clinician-Annotated Benchmark Dataset for Dermatology Visual Question Answering and Reasoning
  2. 2025Derm1M: A Million-scale Vision-Language Dataset Aligned with Clinical Ontology Knowledge for Dermatology
  3. 2024SkinCaRe: A Multimodal Dermatology Dataset Annotated with Medical Caption and Chain-of-Thought Reasoning
  4. 2025DermaSynth: Rich Synthetic Image-Text Pairs Using Open Access Dermatology Datasets
  5. 2025MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment
  6. 2025CLARIFY: A Specialist-Generalist Framework for Accurate and Lightweight Dermatological Visual Question Answering
  7. 2025DermaVQA-DAS: Dermatology Assessment Schema (DAS) & Datasets for Closed-Ended Question Answering & Segmentation in Patient-Generated Dermatology Images
  8. 2025VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosis