# DermoGPT: Open Weights and Open Data for Morphology-Grounded Dermatological Reasoning MLLMs

arXiv: https://arxiv.org/abs/2601.01868 · HTML: https://mlsanity.com/paper/2601.01868/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 5 Jan 2026.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)
- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 21 of 21)

- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale](https://mlsanity.com/paper/2406.19280/) (2024)
- [Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset](https://mlsanity.com/paper/2104.09957/) (2021)
- [Disparities in Dermatology AI: Assessments Using Diverse Clinical Images](https://mlsanity.com/paper/2111.08006/) (2021)
- [PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones](https://mlsanity.com/paper/2007.00478/) (2020)
- [In Search of Lost Online Test-time Adaptation: A Survey](https://mlsanity.com/paper/2310.20199/) (2023)
- [Derm1M: A Million-scale Vision-Language Dataset Aligned with Clinical Ontology Knowledge for Dermatology](https://mlsanity.com/paper/2503.14911/) (2025)
- [Crowdsourcing Dermatology Images with Google Search Ads: Creating a Real-World Skin Condition Dataset](https://mlsanity.com/paper/2402.18545/) (2024)
- [VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced with Clinical Knowledge](https://mlsanity.com/paper/2408.02865/) (2024)
- [MM-Skin: Enhancing Dermatology Vision-Language Model with an Image-Text Dataset Derived from Textbooks](https://mlsanity.com/paper/2505.06152/) (2025)
- [EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysis](https://mlsanity.com/paper/2505.23601/) (2025)
- [MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment](https://mlsanity.com/paper/2505.09372/) (2025)
- [Do we really have to filter out random noise in pre-training data for language models?](https://mlsanity.com/paper/2502.06604/) (2025)
- [DERM12345: A Large, Multisource Dermatoscopic Skin Lesion Dataset with 38 Subclasses](https://mlsanity.com/paper/2406.07426/) (2024)
- [Imbalanced Open Set Domain Adaptation via Moving-threshold Estimation and Gradual Alignment](https://mlsanity.com/paper/2303.04393/) (2023)
- [Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation](https://mlsanity.com/paper/2506.12609/) (2025)
- [ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors](https://mlsanity.com/paper/2502.14627/) (2025)
- [Skin-R1: Clinical Knowledge-Guided Dermatological Diagnosis Using Vision-Language Models](https://mlsanity.com/paper/2511.14900/) (2025)
- [Trustworthy and Fair SkinGPT-R1 for Democratizing Dermatological Reasoning across Diverse Ethnicities](https://mlsanity.com/paper/2511.15242/) (2025)
- [Towards Trustworthy Dermatology MLLMs: A Benchmark and Multimodal Evaluator for Diagnostic Narratives](https://mlsanity.com/paper/2511.09195/) (2025)

## Built on by (most cited first, 3 of 3)

- [MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning](https://mlsanity.com/paper/2602.03320/) (2026)
- [Are Multimodal LLMs Ready for Clinical Dermatology? A Real-World Evaluation in Dermatology](https://mlsanity.com/paper/2605.04098/) (2026)
- [DermAgent: A Self-Reflective Agentic System for Dermatological Image Analysis with Multi-Tool Reasoning and Traceable Decision-Making](https://mlsanity.com/paper/2605.14403/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) show promise for medical applications, yet progress in dermatology lags due to limited training data, narrow task coverage, and lack of clinically-grounded supervision that mirrors expert diagnostic workflows. We present a comprehensive framework to address these gaps. First, we introduce DermoInstruct, a large-scale morphology-anchored instruction corpus comprising 211,243 images and 772,675 trajectories across five task formats, capturing the complete diagnostic pipeline from morphological observation and clinical reasoning to final diagnosis. Second, we establish DermoBench, a rigorous benchmark evaluating 11 tasks across four clinical axes: Morphology, Diagnosis, Reasoning, and Fairness, including a challenging subset of 3,600 expert-verified open-ended instances and human performance baselines. Third, we develop DermoGPT, a dermatology reasoning MLLM trained via supervised fine-tuning followed by our Morphologically-Anchored Visual-Inference-Consistent (MAVIC) reinforcement learning objective, which enforces consistency between visual observations and diagnostic conclusions. At inference, we deploy Confidence-Consistency Test-time adaptation (CCT) for robust predictions. Experiments show DermoGPT significantly outperforms 16 representative baselines across all axes, achieving state-of-the-art performance while substantially narrowing the human-AI gap. DermoInstruct, DermoBench and DermoGPT will be made publicly available at https://github.com/mendicant04/DermoGPT upon acceptance.