# ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions

arXiv: https://arxiv.org/abs/2201.01249 · HTML: https://mlsanity.com/paper/2201.01249/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 4 Jan 2022.

## Builds on (most cited first, 25 of 25)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Understanding Deep Networks via Extremal Perturbations and Smooth Masks](https://mlsanity.com/paper/1910.08485/) (2019)
- [BCN20000: Dermoscopic Lesions in the Wild](https://mlsanity.com/paper/1908.02288/) (2019)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Grounding Visual Explanations](https://mlsanity.com/paper/1807.09685/) (2018)
- [CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation](https://mlsanity.com/paper/2009.10549/) (2020)
- [CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray images](https://mlsanity.com/paper/2004.04931/) (2020)
- [MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network](https://mlsanity.com/paper/1707.02485/) (2017)
- [NiftyNet: a deep-learning platform for medical imaging](https://mlsanity.com/paper/1709.03485/) (2017)
- [On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors](https://mlsanity.com/paper/2005.02000/) (2020)
- [Deep neural network or dermatologist?](https://mlsanity.com/paper/1908.06612/) (2019)
- [Explaining AI-based Decision Support Systems using Concept Localization Maps](https://mlsanity.com/paper/2005.01399/) (2020)
- [The Effects of Skin Lesion Segmentation on the Performance of Dermatoscopic Image Classification](https://mlsanity.com/paper/2008.12602/) (2020)
- [Weakly-Supervised Learning-Based Feature Localization in Confocal Laser Endomicroscopy Glioma Images](https://mlsanity.com/paper/1804.09428/) (2018)
- [Fully Convolutional Neural Networks to Detect Clinical Dermoscopic Features](https://mlsanity.com/paper/1703.04559/) (2017)
- [Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems](https://mlsanity.com/paper/2011.13169/) (2020)
- [The Skincare project, an interactive deep learning system for differential diagnosis of malignant skin lesions. Technical Report](https://mlsanity.com/paper/2005.09448/) (2020)
- [Dermo-DOCTOR: A framework for concurrent skin lesion detection and recognition using a deep convolutional neural network with end-to-end dual encoders](https://mlsanity.com/paper/2102.01824/) (2021)
- [Deep Learning Based Decision Support for Medicine -- A Case Study on Skin Cancer Diagnosis](https://mlsanity.com/paper/2103.05112/) (2021)

## Built on by (most cited first, 19 of 19)

- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Labeling Neural Representations with Inverse Recognition](https://mlsanity.com/paper/2311.13594/) (2023)
- [Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis](https://mlsanity.com/paper/2304.04579/) (2023)
- [Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions](https://mlsanity.com/paper/2306.05731/) (2023)
- [Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma](https://mlsanity.com/paper/2303.12806/) (2023)
- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Explainable Artificial Intelligence for Medical Applications: A Review](https://mlsanity.com/paper/2412.01829/) (2024)
- [Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Care](https://mlsanity.com/paper/2309.00252/) (2023)
- [A Concept-based Interpretable Model for the Diagnosis of Choroid Neoplasias using Multimodal Data](https://mlsanity.com/paper/2403.05606/) (2024)
- [Towards Human-Understandable Multi-Dimensional Concept Discovery](https://mlsanity.com/paper/2503.18629/) (2025)
- [Dermatologist-like explainable AI enhances melanoma diagnosis accuracy: eye-tracking study](https://mlsanity.com/paper/2409.13476/) (2024)
- [A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural language](https://mlsanity.com/paper/2506.05887/) (2025)
- [What Can We Learn from Inter-Annotator Variability in Skin Lesion Segmentation?](https://mlsanity.com/paper/2508.09381/) (2025)
- [Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2507.07148/) (2025)
- [eXplainable Artificial Intelligence on Medical Images: A Survey](https://mlsanity.com/paper/2305.07511/) (2023)
- [DeepMediX: A Deep Learning-Driven Resource-Efficient Medical Diagnosis Across the Spectrum](https://mlsanity.com/paper/2307.00324/) (2023)
- [Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis](https://mlsanity.com/paper/2507.17860/) (2025)
- [Concept Inconsistency in Dermoscopic Concept Bottleneck Models: A Rough-Set Analysis of the Derm7pt Dataset](https://mlsanity.com/paper/2604.19323/) (2026)
- [Towards reliable multimodal disaster severity assessment through preference optimization and explainable vision-language reasoning](https://mlsanity.com/paper/2609.00879/) (2026)

## Abstract (arXiv)

One principal impediment in the successful deployment of AI-based Computer-Aided Diagnosis (CAD) systems in clinical workflows is their lack of transparent decision making. Although commonly used eXplainable AI methods provide some insight into opaque algorithms, such explanations are usually convoluted and not readily comprehensible except by highly trained experts. The explanation of decisions regarding the malignancy of skin lesions from dermoscopic images demands particular clarity, as the underlying medical problem definition is itself ambiguous. This work presents ExAID (Explainable AI for Dermatology), a novel framework for biomedical image analysis, providing multi-modal concept-based explanations consisting of easy-to-understand textual explanations supplemented by visual maps justifying the predictions. ExAID relies on Concept Activation Vectors to map human concepts to those learnt by arbitrary Deep Learning models in latent space, and Concept Localization Maps to highlight concepts in the input space. This identification of relevant concepts is then used to construct fine-grained textual explanations supplemented by concept-wise location information to provide comprehensive and coherent multi-modal explanations. All information is comprehensively presented in a diagnostic interface for use in clinical routines. An educational mode provides dataset-level explanation statistics and tools for data and model exploration to aid medical research and education. Through rigorous quantitative and qualitative evaluation of ExAID, we show the utility of multi-modal explanations for CAD-assisted scenarios even in case of wrong predictions. We believe that ExAID will provide dermatologists an effective screening tool that they both understand and trust. Moreover, it will be the basis for similar applications in other biomedical imaging fields.