mlsanityData to 7 Oct 2026

Explainable Machine Learning with Prior Knowledge: An Overview

arXiv 2105.10172 ↗First posted 21 May 2021cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

This survey presents an overview of integrating prior knowledge into machine learning systems in order to improve explainability. The complexity of machine learning models has elicited research to make them more explainable. However, most explainability methods cannot provide insight beyond the given data, requiring additional information about the context. We propose to harness prior knowledge to improve upon the explanation capabilities of machine learning models. In this paper, we present a categorization of current research into three main categories which either integrate knowledge into the machine learning pipeline, into the explainability method or derive knowledge from explanations. To classify the papers, we build upon the existing taxonomy of informed machine learning and extend it from the perspective of explainability. We conclude with open challenges and research directions.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 49 of its 85 references that are in the map. First posted on arXiv 21 May 2021.

Builds on

49 of 85 references.

References · 49

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  6. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  7. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  8. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 612
  9. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  10. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  11. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  12. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
37 more
  1. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  2. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  3. 2020A Survey on the Explainability of Supervised Machine Learningcited by 225
  4. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  5. 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 203
  6. 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 202
  7. 2016Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layerscited by 197
  8. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 189
  9. 2016Theory-guided Data Science: A New Paradigm for Scientific Discovery from Datacited by 169
  10. 2019Explainable Machine Learning for Scientific Insights and Discoveriescited by 165
  11. 2020Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claimscited by 163
  12. 2020Multi-Objective Counterfactual Explanationscited by 157
  13. 2016GRAM: Graph-based Attention Model for Healthcare Representation Learningcited by 152
  14. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 148
  15. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 144
  16. 2019Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifierscited by 143
  17. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  18. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 138
  19. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 127
  20. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  21. 2019Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approachescited by 101
  22. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  23. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  24. 2017What Does Explainable AI Really Mean? A New Conceptualization of Perspectivescited by 72
  25. 2019Incorporating Priors with Feature Attribution on Text Classificationcited by 68
  26. 2020One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparencycited by 64
  27. 2018Explanation Methods in Deep Learning: Users, Values, Concerns and Challengescited by 49
  28. 2019Jointly Learning Explainable Rules for Recommendation with Knowledge Graphcited by 40
  29. 2017Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspectivecited by 36
  30. 2019Enriching Visual with Verbal Explanations for Relational Concepts -- Combining LIME with Alephcited by 23
  31. 2015Understanding Intra-Class Knowledge Inside CNNcited by 22
  32. 2019Holistic and Comprehensive Annotation of Clinically Significant Findings on Diverse CT Images: Learning from Radiology Reports and Label Ontologycited by 18
  33. 2020Directions for Explainable Knowledge-Enabled Systemscited by 10
  34. 2020Teaching the Machine to Explain Itself using Domain Knowledgecited by 6
  35. 2019Incorporating Biological Knowledge with Factor Graph Neural Network for Interpretable Deep Learningcited by 5
  36. 2019From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)cited by 3
  37. 2017Interactively Transferring CNN Patterns for Part Localizationcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 4; 2023: 5; 2024: 1; 2025: 1.

14 papers.

Sort:

By area:

  1. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  2. 2022A Survey on Explainable Anomaly Detectioncited by 39
  3. 2024A survey on Concept-based Approaches For Model Improvementcited by 8
  4. 2023Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbookcited by 5
  5. 2021Explainable AI for Engineering Design: A Unified Approach of Systems Engineering and Component- Based Deep Learning Demonstrated by Energy- Efficient Building Designcited by 3
  6. 2022Learning to Ignore Adversarial Attackscited by 2
  7. 2023Reason to explain: Interactive contrastive explanations (REASONX)cited by 2
  8. 2021Does Explainable Machine Learning Uncover the Black Box in Vision Applications?cited by 1
  9. 2021Explaining Deep Learning Representations by Tracing the Training Processcited by 0
  10. 2022Explainability in Practice: Estimating Electrification Rates from Mobile Phone Data in Senegalcited by 0
  11. 2023Constructing Word-Context-Coupled Space Aligned with Associative Knowledge Relations for Interpretable Language Modelingcited by 0
  12. 2023Declarative Reasoning on Explanations Using Constraint Logic Programmingcited by 0
2 more
  1. 2023Causal Reasoning through Two Layers of Cognition for Improving Generalization in Visual Question Answeringcited by 0
  2. 2025AxBERT: An Interpretable Chinese Spelling Correction Method Driven by Associative Knowledge Networkcited by 0