# Explaining AI-based Decision Support Systems using Concept Localization Maps

arXiv: https://arxiv.org/abs/2005.01399 · HTML: https://mlsanity.com/paper/2005.01399/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2020.

## Built on by (most cited first, 16 of 16)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis](https://mlsanity.com/paper/2304.04579/) (2023)
- [Concept Embedding Analysis: A Review](https://mlsanity.com/paper/2203.13909/) (2022)
- [ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions](https://mlsanity.com/paper/2201.01249/) (2022)
- [Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?](https://mlsanity.com/paper/2409.13456/) (2024)
- [Privacy Meets Explainability: A Comprehensive Impact Benchmark](https://mlsanity.com/paper/2211.04110/) (2022)
- [Generating Counterfactual Trajectories with Latent Diffusion Models for Concept Discovery](https://mlsanity.com/paper/2404.10356/) (2024)
- [Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces](https://mlsanity.com/paper/2311.14435/) (2023)
- [Revealing Similar Semantics Inside CNNs: An Interpretable Concept-based Comparison of Feature Spaces](https://mlsanity.com/paper/2305.07663/) (2023)
- [Deep Learning Based Decision Support for Medicine -- A Case Study on Skin Cancer Diagnosis](https://mlsanity.com/paper/2103.05112/) (2021)
- [FovEx: Human-Inspired Explanations for Vision Transformers and Convolutional Neural Networks](https://mlsanity.com/paper/2408.02123/) (2024)
- [CE-FAM: Concept-Based Explanation via Fusion of Activation Maps](https://mlsanity.com/paper/2509.23849/) (2025)
- [Discovering Concept Directions from Diffusion-based Counterfactuals via Latent Clustering](https://mlsanity.com/paper/2505.07073/) (2025)
- [Probing the Probes: Methods and Metrics for Concept Alignment](https://mlsanity.com/paper/2511.04312/) (2025)
- [E-TCAV: Formalizing Penultimate Proxies for Efficient Concept Based Interpretability](https://mlsanity.com/paper/2605.10261/) (2026)

## Abstract (arXiv)

Human-centric explainability of AI-based Decision Support Systems (DSS) using visual input modalities is directly related to reliability and practicality of such algorithms. An otherwise accurate and robust DSS might not enjoy trust of experts in critical application areas if it is not able to provide reasonable justification of its predictions. This paper introduces Concept Localization Maps (CLMs), which is a novel approach towards explainable image classifiers employed as DSS. CLMs extend Concept Activation Vectors (CAVs) by locating significant regions corresponding to a learned concept in the latent space of a trained image classifier. They provide qualitative and quantitative assurance of a classifier's ability to learn and focus on similar concepts important for humans during image recognition. To better understand the effectiveness of the proposed method, we generated a new synthetic dataset called Simple Concept DataBase (SCDB) that includes annotations for 10 distinguishable concepts, and made it publicly available. We evaluated our proposed method on SCDB as well as a real-world dataset called CelebA. We achieved localization recall of above 80% for most relevant concepts and average recall above 60% for all concepts using SE-ResNeXt-50 on SCDB. Our results on both datasets show great promise of CLMs for easing acceptance of DSS in practice.