# Knowledge-intensive Language Understanding for Explainable AI

arXiv: https://arxiv.org/abs/2108.01174 · HTML: https://mlsanity.com/paper/2108.01174/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 17 of 17)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems](https://mlsanity.com/paper/1906.10263/) (2019)
- [Semantics of the Black-Box: Can knowledge graphs help make deep learning systems more interpretable and explainable?](https://mlsanity.com/paper/2010.08660/) (2020)
- [Multi-hop Question Generation with Graph Convolutional Network](https://mlsanity.com/paper/2010.09240/) (2020)
- [Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation](https://mlsanity.com/paper/1611.08191/) (2016)
- [KI-BERT: Infusing Knowledge Context for Better Language and Domain Understanding](https://mlsanity.com/paper/2104.08145/) (2021)
- ["Who can help me?": Knowledge Infused Matching of Support Seekers and Support Providers during COVID-19 on Reddit](https://mlsanity.com/paper/2105.06398/) (2021)
- [Knowledge Infused Policy Gradients for Adaptive Pandemic Control](https://mlsanity.com/paper/2102.06245/) (2021)
- [Knowledge Infused Policy Gradients with Upper Confidence Bound for Relational Bandits](https://mlsanity.com/paper/2106.13895/) (2021)

## Built on by (most cited first, 25 of 25)

- [Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems](https://mlsanity.com/paper/2301.11333/) (2023)
- [ISEEQ: Information Seeking Question Generation using Dynamic Meta-Information Retrieval and Knowledge Graphs](https://mlsanity.com/paper/2112.07622/) (2021)
- [Process Knowledge-Infused AI: Towards User-level Explainability, Interpretability, and Safety](https://mlsanity.com/paper/2206.13349/) (2022)
- [An Accurate and Interpretable Framework for Trustworthy Process Monitoring](https://mlsanity.com/paper/2302.10426/) (2023)
- [Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and Safety](https://mlsanity.com/paper/2312.06798/) (2023)
- [Towards Tractable Mathematical Reasoning: Challenges, Strategies, and Opportunities for Solving Math Word Problems](https://mlsanity.com/paper/2111.05364/) (2021)
- [ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance](https://mlsanity.com/paper/2305.08010/) (2023)
- [Process Knowledge-infused Learning for Clinician-friendly Explanations](https://mlsanity.com/paper/2306.09824/) (2023)
- [KSAT: Knowledge-infused Self Attention Transformer -- Integrating Multiple Domain-Specific Contexts](https://mlsanity.com/paper/2210.04307/) (2022)
- [A Domain-Agnostic Neurosymbolic Approach for Big Social Data Analysis: Evaluating Mental Health Sentiment on Social Media during COVID-19](https://mlsanity.com/paper/2411.07163/) (2024)
- [Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions](https://mlsanity.com/paper/2509.02452/) (2025)
- [Knowledge Graph Guided Semantic Evaluation of Language Models For User Trust](https://mlsanity.com/paper/2305.04989/) (2023)
- [Cook-Gen: Robust Generative Modeling of Cooking Actions from Recipes](https://mlsanity.com/paper/2306.01805/) (2023)
- [Knowledge-Infused Self Attention Transformers](https://mlsanity.com/paper/2306.13501/) (2023)
- [IERL: Interpretable Ensemble Representation Learning -- Combining CrowdSourced Knowledge and Distributed Semantic Representations](https://mlsanity.com/paper/2306.13865/) (2023)
- [Stereotype Detection in LLMs: A Multiclass, Explainable, and Benchmark-Driven Approach](https://mlsanity.com/paper/2404.01768/) (2024)
- [Explainable Interface for Human-Autonomy Teaming: A Survey](https://mlsanity.com/paper/2405.02583/) (2024)
- [Can Language Models Capture Graph Semantics? From Graphs to Language Model and Vice-Versa](https://mlsanity.com/paper/2206.09259/) (2022)
- [L3 Ensembles: Lifelong Learning Approach for Ensemble of Foundational Language Models](https://mlsanity.com/paper/2311.06493/) (2023)
- [Neurosymbolic Value-Inspired AI (Why, What, and How)](https://mlsanity.com/paper/2312.09928/) (2023)
- [RDR: the Recap, Deliberate, and Respond Method for Enhanced Language Understanding](https://mlsanity.com/paper/2312.09932/) (2023)
- [Reliability Analysis of Psychological Concept Extraction and Classification in User-penned Text](https://mlsanity.com/paper/2401.06709/) (2024)
- [IoT-Based Preventive Mental Health Using Knowledge Graphs and Standards for Better Well-Being](https://mlsanity.com/paper/2406.13791/) (2024)
- [NeuroLit Navigator: A Neurosymbolic Approach to Scholarly Article Searches for Systematic Reviews](https://mlsanity.com/paper/2503.00278/) (2025)
- [Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment](https://mlsanity.com/paper/2609.07766/) (2026)

## Abstract (arXiv)

AI systems have seen significant adoption in various domains. At the same time, further adoption in some domains is hindered by inability to fully trust an AI system that it will not harm a human. Besides the concerns for fairness, privacy, transparency, and explainability are key to developing trusts in AI systems. As stated in describing trustworthy AI "Trust comes through understanding. How AI-led decisions are made and what determining factors were included are crucial to understand." The subarea of explaining AI systems has come to be known as XAI. Multiple aspects of an AI system can be explained; these include biases that the data might have, lack of data points in a particular region of the example space, fairness of gathering the data, feature importances, etc. However, besides these, it is critical to have human-centered explanations that are directly related to decision-making similar to how a domain expert makes decisions based on "domain knowledge," that also include well-established, peer-validated explicit guidelines. To understand and validate an AI system's outcomes (such as classification, recommendations, predictions), that lead to developing trust in the AI system, it is necessary to involve explicit domain knowledge that humans understand and use.