Locally Testing Model Detections for Semantic Global Concepts
Not on the map. Why?
Abstract (arXiv)
Ensuring the quality of black-box Deep Neural Networks (DNNs) has become ever more significant, especially in safety-critical domains such as automated driving. While global concept encodings generally enable a user to test a model for a specific concept, linking global concept encodings to the local processing of single network inputs reveals their strengths and limitations. Our proposed framework global-to-local Concept Attribution (glCA) uses approaches from local (why a specific prediction originates) and global (how a model works generally) eXplainable Artificial Intelligence (xAI) to test DNNs for a predefined semantical concept locally. The approach allows for conditioning local, post-hoc explanations on predefined semantic concepts encoded as linear directions in the model's latent space. Pixel-exact scoring concerning the global concept usage assists the tester in further understanding the model processing of single data points for the selected concept. Our approach has the advantage of fully covering the model-internal encoding of the semantic concept and allowing the localization of relevant concept-related information. The results show major differences in the local perception and usage of individual global concept encodings and demand for further investigations regarding obtaining thorough semantic concept encodings.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 46 references that are in the map. First posted on arXiv 27 May 2024.
Builds on
31 of 46 references.
References · 31
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
- 2017SmoothGrad: removing noise by adding noisecited by 1,109
- 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
19 more
- 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
- 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
- 2020Concept Whitening for Interpretable Image Recognitioncited by 225
- 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 178
- 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
- 2018Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networkscited by 129
- 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 126
- 2020Invertible Concept-based Explanations for CNN Models with Non-negative Concept Activation Vectorscited by 120
- 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 91
- 2019Towards Best Practice in Explaining Neural Network Decisions with LRPcited by 70
- 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
- 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 55
- 2023Multi-dimensional concept discovery (MCD): A unifying framework with completeness guaranteescited by 50
- 2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
- 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspacescited by 25
- 2022Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanationscited by 17
- 2019Concept Saliency Maps to Visualize Relevant Features in Deep Generative Modelscited by 11
- 2022Optimizing Explanations by Network Canonization and Hyperparameter Searchcited by 11
- 2022Measurably Stronger Explanation Reliability via Model Canonizationcited by 6
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.