# VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosis

arXiv: https://arxiv.org/abs/2508.06624 · HTML: https://mlsanity.com/paper/2508.06624/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 8 Aug 2025.

## Builds on (most cited first, 24 of 24)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2403.16999/) (2024)
- [LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models](https://mlsanity.com/paper/2306.09265/) (2023)
- [Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey](https://mlsanity.com/paper/2412.20367/) (2024)
- [A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine](https://mlsanity.com/paper/2405.08603/) (2024)
- [Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Models](https://mlsanity.com/paper/2410.19732/) (2024)
- [SCORE: Story Coherence and Retrieval Enhancement for AI Narratives](https://mlsanity.com/paper/2503.23512/) (2025)
- [MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration](https://mlsanity.com/paper/2506.19835/) (2025)
- [Towards Robust Ranker for Text Retrieval](https://mlsanity.com/paper/2206.08063/) (2022)
- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation](https://mlsanity.com/paper/2505.24787/) (2025)
- [Fine-Grained Distillation for Long Document Retrieval](https://mlsanity.com/paper/2212.10423/) (2022)
- [Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis](https://mlsanity.com/paper/2304.04579/) (2023)
- [Cross-Modal Consistency in Multimodal Large Language Models](https://mlsanity.com/paper/2411.09273/) (2024)
- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems](https://mlsanity.com/paper/2011.13169/) (2020)
- [A Survey on Image-text Multimodal Models](https://mlsanity.com/paper/2309.15857/) (2023)
- [CBVLM: Training-free Explainable Concept-based Large Vision Language Models for Medical Image Classification](https://mlsanity.com/paper/2501.12266/) (2025)
- [Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis](https://mlsanity.com/paper/2410.15446/) (2024)
- [Guarding the Gate: ConceptGuard Battles Concept-Level Backdoors in Concept Bottleneck Models](https://mlsanity.com/paper/2411.16512/) (2024)
- [Lesion Elevation Prediction from Skin Images Improves Diagnosis](https://mlsanity.com/paper/2408.02792/) (2024)

## Abstract (arXiv)

Accurate diagnosis of skin diseases remains a significant challenge due to the complex and diverse visual features present in dermatoscopic images, often compounded by a lack of interpretability in existing purely visual diagnostic models. To address these limitations, this study introduces VL-MedGuide (Visual-Linguistic Medical Guide), a novel framework leveraging the powerful multi-modal understanding and reasoning capabilities of Visual-Language Large Models (LVLMs) for intelligent and inherently interpretable auxiliary diagnosis of skin conditions. VL-MedGuide operates in two interconnected stages: a Multi-modal Concept Perception Module, which identifies and linguistically describes dermatologically relevant visual features through sophisticated prompt engineering, and an Explainable Disease Reasoning Module, which integrates these concepts with raw visual information via Chain-of-Thought prompting to provide precise disease diagnoses alongside transparent rationales. Comprehensive experiments on the Derm7pt dataset demonstrate that VL-MedGuide achieves state-of-the-art performance in both disease diagnosis (83.55% BACC, 80.12% F1) and concept detection (76.10% BACC, 67.45% F1), surpassing existing baselines. Furthermore, human evaluations confirm the high clarity, completeness, and trustworthiness of its generated explanations, bridging the gap between AI performance and clinical utility by offering actionable, explainable insights for dermatological practice.