Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models
Not on the map. Why?
Abstract (arXiv)
With the availability of large databases and recent improvements in deep learning methodology, the performance of AI systems is reaching or even exceeding the human level on an increasing number of complex tasks. Impressive examples of this development can be found in domains such as image classification, sentiment analysis, speech understanding or strategic game playing. However, because of their nested non-linear structure, these highly successful machine learning and artificial intelligence models are usually applied in a black box manner, i.e., no information is provided about what exactly makes them arrive at their predictions. Since this lack of transparency can be a major drawback, e.g., in medical applications, the development of methods for visualizing, explaining and interpreting deep learning models has recently attracted increasing attention. This paper summarizes recent developments in this field and makes a plea for more interpretability in artificial intelligence. Furthermore, it presents two approaches to explaining predictions of deep learning models, one method which computes the sensitivity of the prediction with respect to changes in the input and one approach which meaningfully decomposes the decision in terms of the input variables. These methods are evaluated on three classification tasks.
Key facts. Cited by 202 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Aug 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2020: 2; 2021: 37; 2022: 53; 2023: 36; 2024: 26; 2025: 34; 2026: 14.
202 papers.
Sort:
By area:
- 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
- 2021Explanations in Autonomous Driving: A Surveycited by 80
- 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
- 2021Evaluating Robustness of Counterfactual Explanationscited by 41
- 2021Explainable Artificial Intelligence Approaches: A Surveycited by 40
- 2022Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directionscited by 34
- 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
- 2024Backward Lens: Projecting Language Model Gradients into the Vocabulary Spacecited by 30
- 2022The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizationscited by 29
- 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunitiescited by 25
- 2022WindowSHAP: An Efficient Framework for Explaining Time-series Classifiers based on Shapley Valuescited by 24
38 more
- 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Modelscited by 23
- 2022The Role of Machine Learning in Cybersecuritycited by 21
- 2022Machine Learning Security in Industry: A Quantitative Surveycited by 21
- 2021A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Datacited by 19
- 2022Learning When to Advise Human Decision Makerscited by 18
- 2022Interpreting Unfairness in Graph Neural Networks via Training Node Attributioncited by 18
- 2023XAIR: A Framework of Explainable AI in Augmented Realitycited by 17
- 2021Can Explainable AI Explain Unfairness? A Framework for Evaluating Explainable AIcited by 16
- 2022Attribution-based XAI Methods in Computer Vision: A Reviewcited by 16
- 2021Explainable Autonomous Robots: A Survey and Perspectivecited by 15
- 2022Towards Explainable Evaluation Metrics for Natural Language Generationcited by 15
- 2023Towards Explainable Evaluation Metrics for Machine Translationcited by 15
- 2021A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networkscited by 14
- 2021Explainable Machine Learning with Prior Knowledge: An Overviewcited by 14
- 2021Explainable Artificial Intelligence for Human Decision-Support System in Medical Domaincited by 12
- 2024Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithmscited by 12
- 2021Expressive Explanations of DNNs by Combining Concept Analysis with ILPcited by 11
- 2021Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extractioncited by 11
- 2021Interpretable Approximation of High-Dimensional Datacited by 10
- 2022A Measure of the Complexity of Neural Representations based on Partial Information Decompositioncited by 10
- 2022SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevancecited by 9
- 2022Causality-Inspired Taxonomy for Explainable Artificial Intelligencecited by 9
- 2022Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Searchcited by 9
- 2023Concept Distillation: Leveraging Human-Centered Explanations for Model Improvementcited by 9
- 2025Artificial intelligence for representing and characterizing quantum systemscited by 9
- 2021Interpreting Deep Learning based Cerebral Palsy Prediction with Channel Attentioncited by 8
- 2021Generating Contrastive Explanations for Inductive Logic Programming Based on a Near Miss Approachcited by 8
- 2022Scenario-Assisted Deep Reinforcement Learningcited by 8
- 2022Explainable Artificial Intelligence (XAI) for Internet of Things: A Surveycited by 8
- 2022Ask to Know More: Generating Counterfactual Explanations for Fake Claimscited by 8
- 2022Generating and Detecting True Ambiguity: A Forgotten Danger in DNN Supervision Testingcited by 8
- 2023A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functionscited by 8
- 2023An Attribution Method for Siamese Encoderscited by 8
- 2024KAN You See It? KANs and Sentinel for Effective and Explainable Crop Field Segmentationcited by 8
- 2024Bridging Interpretability and Robustness Using LIME-Guided Model Refinementcited by 8
- 2021Feature selection revisited in the single-cell eracited by 7
- 2022Explaining Reject Options of Learning Vector Quantization Classifierscited by 7
- 2022Explain to Not Forget: Defending Against Catastrophic Forgetting with XAIcited by 7
50 most cited of 202 All 202 (JSON)
- 2026A Unifying Framework of Concept-based Explainable AI with Completeness Guarantees28 Sep 2026
- 2026Template Ageing and Longitudinal Verification in Fixed-Text Keystroke Dynamics: A Subject-Disjoint Study Across Eight Weeks24 Sep 2026
- 2026Machine Learning in Fish Farming12 Sep 2026
- 2026XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?8 Sep 2026
- 2026Explainable Diabetic Retinopathy Classification Using Vision Foundation Models28 Aug 2026
- 2026Beyond Accuracy: Evaluating Efficiency, Robustness and Explainability in Deep Learning for Malaria Diagnosis29 May 2026
- 2026Systematic Evaluation of Vision Transformers for Automated Cervical Cancer Classification: Optimization, Statistical Validation, and Clinical Interpretability17 May 2026
- 2026Explainability of Recurrent Neural Networks for Enhancing P300-based Brain-Computer Interfaces11 May 2026
- 2026Explanation-Aware Learning for Enhanced Interpretability in Biomedical Imaging11 May 2026
- 2026Fusion-CAM: Integrating Gradient and Region-Based Class Activation Maps for Robust Visual Explanations5 Mar 2026
- 2026Fair feature attribution for multi-output prediction: a Shapley-based perspective26 Feb 2026
- 2026CLEAR-HPV: Interpretable concept discovery for human-papillomavirus-associated morphology in whole-slide histology4 Feb 2026
38 more
- 2026Explaining Machine Learning Predictive Models through Conditional Expectation Methods12 Jan 2026
- 2026Explainability-Guided Defense: Attribution-Aware Model Refinement Against Adversarial Data Attacks2 Jan 2026
- 2025Faithful and Stable Neuron Explanations for Trustworthy Mechanistic Interpretability19 Dec 2025
- 2025Explanation Beyond Intuition: A Testable Criterion for Inherent Explainability19 Dec 2025
- 2025Machine Learning-Based Basil Yield Prediction in IoT-Enabled Indoor Vertical Hydroponic Farms15 Dec 2025
- 2025Refining Visual Artifacts in Diffusion Models via Explainable AI-based Flaw Activation Maps9 Dec 2025
- 2025WalkCLIP: Multimodal Learning for Urban Walkability Prediction26 Nov 2025
- 2025Generation, Evaluation, and Explanation of Novelists' Styles with Single-Token Prompts25 Nov 2025
- 2025Accuracy is Not Enough: Poisoning Interpretability in Federated Learning via Color Skew17 Nov 2025
- 2025Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challenges27 Oct 2025
- 2025Explainable Face Presentation Attack Detection via Ensemble-CAM22 Oct 2025
- 2025FST.ai 2.0: An Explainable AI Ecosystem for Fair, Fast, and Inclusive Decision-Making in Olympic and Paralympic Taekwondo21 Oct 2025
- 2025Identifying Information-Transfer Nodes in a Recurrent Neural Network Reveals Dynamic Representations29 Sep 2025
- 2025Towards a Transparent and Interpretable AI Model for Medical Image Classifications20 Sep 2025
- 2025Artificial intelligence for representing and characterizing quantum systems5 Sep 2025
- 2025Segmentation and Classification of Pap Smear Images for Cervical Cancer Detection Using Deep Learning25 Aug 2025
- 2025X-Node: Self-Explanation is All We Need14 Aug 2025
- 2025CTG-Insight: A Multi-Agent Interpretable LLM Framework for Cardiotocography Analysis and Classification29 Jul 2025
- 2025TriGuard: Testing Model Safety with Attribution Entropy, Verification, and Drift17 Jun 2025
- 2025Domain Adaptation for Big Data in Agricultural Image Analysis: A Comprehensive Review6 Jun 2025
- 2025A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural language6 Jun 2025
- 2025Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability2 Jun 2025
- 2025Explaining Large Language Models with gSMILE27 May 2025
- 2025Transformers in Protein: A Survey26 May 2025
- 2025An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations22 May 2025
- 2025Enhancing Vision Transformer Explainability Using Artificial Astrocytes20 May 2025
- 2025Fixed Point Explainability18 May 2025
- 2025PRUNE: A Patching Based Repair Framework for Certifiable Unlearning of Neural Networks10 May 2025
- 2025Engineering Risk-Aware, Security-by-Design Frameworks for Assurance of Large-Scale Autonomous AI Models9 May 2025
- 2025Explainable Face Recognition via Improved Localization4 May 2025
- 2025Financial Fraud Detection with Entropy Computing14 Mar 2025
- 2025VirtualXAI: A User-Centric Framework for Explainability Assessment Leveraging GPT-Generated Personas6 Mar 2025
- 2025Document clustering with evolved multiword search queries24 Feb 2025
- 2025AI-Assisted Decision Making with Human Learning18 Feb 2025
- 2025Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models30 Jan 2025
- 2025Faithful Counterfactual Visual Explanations (FCVE)12 Jan 2025
- 2024Reconciling Privacy and Explainability in High-Stakes: A Systematic Inquiry30 Dec 2024
- 2024Bridging Interpretability and Robustness Using LIME-Guided Model Refinement25 Dec 2024
50 newest of 202 All 202 (JSON)
- 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
- 2021Explanations in Autonomous Driving: A Surveycited by 80
- 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
- 2021Evaluating Robustness of Counterfactual Explanationscited by 41
- 2021Explainable Artificial Intelligence Approaches: A Surveycited by 40
- 2022Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directionscited by 34
- 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
- 2024Backward Lens: Projecting Language Model Gradients into the Vocabulary Spacecited by 30
- 2022The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizationscited by 29
- 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunitiescited by 25
- 2022WindowSHAP: An Efficient Framework for Explaining Time-series Classifiers based on Shapley Valuescited by 24
12 most cited of 196 All 196 (JSON)
- 2026A Unifying Framework of Concept-based Explainable AI with Completeness Guarantees28 Sep 2026
- 2026Template Ageing and Longitudinal Verification in Fixed-Text Keystroke Dynamics: A Subject-Disjoint Study Across Eight Weeks24 Sep 2026
- 2026Machine Learning in Fish Farming12 Sep 2026
- 2026XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?8 Sep 2026
- 2026Explainable Diabetic Retinopathy Classification Using Vision Foundation Models28 Aug 2026
- 2026Beyond Accuracy: Evaluating Efficiency, Robustness and Explainability in Deep Learning for Malaria Diagnosis29 May 2026
- 2026Systematic Evaluation of Vision Transformers for Automated Cervical Cancer Classification: Optimization, Statistical Validation, and Clinical Interpretability17 May 2026
- 2026Explainability of Recurrent Neural Networks for Enhancing P300-based Brain-Computer Interfaces11 May 2026
- 2026Explanation-Aware Learning for Enhanced Interpretability in Biomedical Imaging11 May 2026
- 2026Fusion-CAM: Integrating Gradient and Region-Based Class Activation Maps for Robust Visual Explanations5 Mar 2026
- 2026Fair feature attribution for multi-output prediction: a Shapley-based perspective26 Feb 2026
- 2026CLEAR-HPV: Interpretable concept discovery for human-papillomavirus-associated morphology in whole-slide histology4 Feb 2026
12 newest of 196 All 196 (JSON)