# Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis

arXiv: https://arxiv.org/abs/2304.04579 · HTML: https://mlsanity.com/paper/2304.04579/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [A deep learning system for differential diagnosis of skin diseases](https://mlsanity.com/paper/1909.05382/) (2019)
- [Now You See Me (CME): Concept-based Model Extraction](https://mlsanity.com/paper/2010.13233/) (2020)
- [A Framework for Learning Ante-hoc Explainable Models via Concepts](https://mlsanity.com/paper/2108.11761/) (2021)
- [On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors](https://mlsanity.com/paper/2005.02000/) (2020)
- [Regression Concept Vectors for Bidirectional Explanations in Histopathology](https://mlsanity.com/paper/1904.04520/) (2019)
- [The Doctor Just Won't Accept That!](https://mlsanity.com/paper/1711.08037/) (2017)
- [ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions](https://mlsanity.com/paper/2201.01249/) (2022)
- [Explaining AI-based Decision Support Systems using Concept Localization Maps](https://mlsanity.com/paper/2005.01399/) (2020)
- [Comprehensible Convolutional Neural Networks via Guided Concept Learning](https://mlsanity.com/paper/2101.03919/) (2021)
- [Deep Sequential Feature Learning in Clinical Image Classification of Infectious Keratitis](https://mlsanity.com/paper/2006.02666/) (2020)

## Built on by (most cited first, 26 of 26)

- [MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment](https://mlsanity.com/paper/2401.08527/) (2024)
- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Concept-Attention Whitening for Interpretable Skin Lesion Diagnosis](https://mlsanity.com/paper/2404.05997/) (2024)
- [A Framework for Interpretability in Machine Learning for Medical Imaging](https://mlsanity.com/paper/2310.01685/) (2023)
- [Towards Concept-based Interpretability of Skin Lesion Diagnosis using Vision-Language Models](https://mlsanity.com/paper/2311.14339/) (2023)
- [Aligning Medical Images with General Knowledge from Large Language Models](https://mlsanity.com/paper/2409.00341/) (2024)
- [CBVLM: Training-free Explainable Concept-based Large Vision Language Models for Medical Image Classification](https://mlsanity.com/paper/2501.12266/) (2025)
- [Incorporating Expert Rules into Neural Networks in the Framework of Concept-Based Learning](https://mlsanity.com/paper/2402.14726/) (2024)
- [Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis](https://mlsanity.com/paper/2410.15446/) (2024)
- [LISArD: Learning Image Similarity to Defend Against Gray-box Adversarial Attacks](https://mlsanity.com/paper/2502.20562/) (2025)
- [A Novel Perspective for Multi-modal Multi-label Skin Lesion Classification](https://mlsanity.com/paper/2409.12390/) (2024)
- [What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?](https://mlsanity.com/paper/2508.09381/) (2025)
- [MAProtoNet: A Multi-scale Attentive Interpretable Prototypical Part Network for 3D Magnetic Resonance Imaging Brain Tumor Classification](https://mlsanity.com/paper/2404.08917/) (2024)
- [FI-CBL: A Probabilistic Method for Concept-Based Learning with Expert Rules](https://mlsanity.com/paper/2406.19897/) (2024)
- [ParseCaps: An Interpretable Parsing Capsule Network for Medical Image Diagnosis](https://mlsanity.com/paper/2411.01564/) (2024)
- [Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2507.07148/) (2025)
- [ZQBA: Zero Query Black-box Adversarial Attack](https://mlsanity.com/paper/2510.00769/) (2025)
- [CFCML: A Coarse-to-Fine Crossmodal Learning Framework For Disease Diagnosis Using Multimodal Images and Tabular Data](https://mlsanity.com/paper/2603.20016/) (2026)
- [Uncertainty-Aware Information Pursuit for Interpretable and Reliable Medical Image Analysis](https://mlsanity.com/paper/2506.16742/) (2025)
- [VL-MedGuide: A Visual-Linguistic Large Model for Intelligent and Explainable Skin Disease Auxiliary Diagnosis](https://mlsanity.com/paper/2508.06624/) (2025)
- [ConceptFlow: Hierarchical and Fine-grained Concept-Based Explanation for Convolutional Neural Networks](https://mlsanity.com/paper/2509.18147/) (2025)
- [CoPA: Hierarchical Concept Prompting and Aggregating Network for Explainable Diagnosis](https://mlsanity.com/paper/2510.03767/) (2025)
- [ViConEx-Med: Visual Concept Explainability via Multi-Concept Token Transformer for Medical Image Analysis](https://mlsanity.com/paper/2510.10174/) (2025)
- [Concept Inconsistency in Dermoscopic Concept Bottleneck Models: A Rough-Set Analysis of the Derm7pt Dataset](https://mlsanity.com/paper/2604.19323/) (2026)
- [Context-driven Missing-Modality Learning for Robust Medical Diagnosis with Image-Tabular Data](https://mlsanity.com/paper/2605.25968/) (2026)
- [Beyond Heatmaps: Unsupervised Concept-Graph Reasoning for Interpretable Visual Explanation](https://mlsanity.com/paper/2607.01416/) (2026)

## Abstract (arXiv)

Early detection of melanoma is crucial for preventing severe complications and increasing the chances of successful treatment. Existing deep learning approaches for melanoma skin lesion diagnosis are deemed black-box models, as they omit the rationale behind the model prediction, compromising the trustworthiness and acceptability of these diagnostic methods. Attempts to provide concept-based explanations are based on post-hoc approaches, which depend on an additional model to derive interpretations. In this paper, we propose an inherently interpretable framework to improve the interpretability of concept-based models by incorporating a hard attention mechanism and a coherence loss term to assure the visual coherence of concept activations by the concept encoder, without requiring the supervision of additional annotations. The proposed framework explains its decision in terms of human-interpretable concepts and their respective contribution to the final prediction, as well as a visual interpretation of the locations where the concept is present in the image. Experiments on skin image datasets demonstrate that our method outperforms existing black-box and concept-based models for skin lesion classification.