# On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors

arXiv: https://arxiv.org/abs/2005.02000 · HTML: https://mlsanity.com/paper/2005.02000/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2020.

## Built on by (most cited first, 30 of 39)

- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)](https://mlsanity.com/paper/2402.10376/) (2024)
- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Learning Concise and Descriptive Attributes for Visual Recognition](https://mlsanity.com/paper/2308.03685/) (2023)
- [Interpreting CLIP with Hierarchical Sparse Autoencoders](https://mlsanity.com/paper/2502.20578/) (2025)
- [SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysis](https://mlsanity.com/paper/2302.00785/) (2023)
- [Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decision](https://mlsanity.com/paper/2303.00885/) (2023)
- [Using Sequences of Life-events to Predict Human Lives](https://mlsanity.com/paper/2306.03009/) (2023)
- [Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis](https://mlsanity.com/paper/2304.04579/) (2023)
- [Red Teaming Deep Neural Networks with Feature Synthesis Tools](https://mlsanity.com/paper/2302.10894/) (2023)
- [ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions](https://mlsanity.com/paper/2201.01249/) (2022)
- [Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma](https://mlsanity.com/paper/2303.12806/) (2023)
- [MICA: Towards Explainable Skin Lesion Diagnosis via Multi-Level Image-Concept Alignment](https://mlsanity.com/paper/2401.08527/) (2024)
- [The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations](https://mlsanity.com/paper/2307.00364/) (2023)
- [Visual-TCAV: Concept-based Attribution and Saliency Maps for Post-hoc Explainability in Image Classification](https://mlsanity.com/paper/2411.05698/) (2024)
- [Explainable Artificial Intelligence for Medical Applications: A Review](https://mlsanity.com/paper/2412.01829/) (2024)
- [Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat](https://mlsanity.com/paper/2307.05350/) (2023)
- [Making Corgis Important for Honeycomb Classification: Adversarial Attacks on Concept-based Explainability Tools](https://mlsanity.com/paper/2110.07120/) (2021)
- [Towards Concept-based Interpretability of Skin Lesion Diagnosis using Vision-Language Models](https://mlsanity.com/paper/2311.14339/) (2023)
- [Interpretability for Multimodal Emotion Recognition using Concept Activation Vectors](https://mlsanity.com/paper/2202.01072/) (2022)
- [Benchmarking and Enhancing Disentanglement in Concept-Residual Models](https://mlsanity.com/paper/2312.00192/) (2023)
- [Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Care](https://mlsanity.com/paper/2309.00252/) (2023)
- [Learning Unsupervised Hierarchies of Audio Concepts](https://mlsanity.com/paper/2207.11231/) (2022)
- [Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis](https://mlsanity.com/paper/2410.15446/) (2024)
- [Tackling Shortcut Learning in Deep Neural Networks: An Iterative Approach with Interpretable Models](https://mlsanity.com/paper/2302.10289/) (2023)
- [Exploring the Lottery Ticket Hypothesis with Explainability Methods: Insights into Sparse Network Performance](https://mlsanity.com/paper/2307.13698/) (2023)
- [Unsupervised discovery of Interpretable Visual Concepts](https://mlsanity.com/paper/2309.00018/) (2023)
- [Concept-based Explainable Malignancy Scoring on Pulmonary Nodules in CT Images](https://mlsanity.com/paper/2405.17483/) (2024)
- [Explaining Human Activity Recognition with SHAP: Validating Insights with Perturbation and Quantitative Measures](https://mlsanity.com/paper/2411.03714/) (2024)

## Abstract (arXiv)

Deep learning based medical image classifiers have shown remarkable prowess in various application areas like ophthalmology, dermatology, pathology, and radiology. However, the acceptance of these Computer-Aided Diagnosis (CAD) systems in real clinical setups is severely limited primarily because their decision-making process remains largely obscure. This work aims at elucidating a deep learning based medical image classifier by verifying that the model learns and utilizes similar disease-related concepts as described and employed by dermatologists. We used a well-trained and high performing neural network developed by REasoning for COmplex Data (RECOD) Lab for classification of three skin tumours, i.e. Melanocytic Naevi, Melanoma and Seborrheic Keratosis and performed a detailed analysis on its latent space. Two well established and publicly available skin disease datasets, PH2 and derm7pt, are used for experimentation. Human understandable concepts are mapped to RECOD image classification model with the help of Concept Activation Vectors (CAVs), introducing a novel training and significance testing paradigm for CAVs. Our results on an independent evaluation set clearly shows that the classifier learns and encodes human understandable concepts in its latent representation. Additionally, TCAV scores (Testing with CAVs) suggest that the neural network indeed makes use of disease-related concepts in the correct way when making predictions. We anticipate that this work can not only increase confidence of medical practitioners on CAD but also serve as a stepping stone for further development of CAV-based neural network interpretation methods.