# Explanation Ontology: A Model of Explanations for User-Centered AI

arXiv: https://arxiv.org/abs/2010.01479 · HTML: https://mlsanity.com/paper/2010.01479/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2020.

## Built on by (most cited first, 13 of 13)

- [Principles of Explanation in Human-AI Systems](https://mlsanity.com/paper/2102.04972/) (2021)
- [Knowledge Graphs for the Life Sciences: Recent Developments, Challenges and Opportunities](https://mlsanity.com/paper/2309.17255/) (2023)
- [Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes](https://mlsanity.com/paper/2302.05752/) (2023)
- [How Mature is Requirements Engineering for AI-based Systems? A Systematic Mapping Study on Practices, Challenges, and Future Research Directions](https://mlsanity.com/paper/2409.07192/) (2024)
- [Tell me more: Intent Fulfilment Framework for Enhancing User Experiences in Conversational XAI](https://mlsanity.com/paper/2405.10446/) (2024)
- [Semantic Modeling for Food Recommendation Explanations](https://mlsanity.com/paper/2105.01269/) (2021)
- [Trust, Accountability, and Autonomy in Knowledge Graph-based AI for Self-determination](https://mlsanity.com/paper/2310.19503/) (2023)
- [Modeling the Quality of Dialogical Explanations](https://mlsanity.com/paper/2403.00662/) (2024)
- [Knowledge-based XAI through CBR: There is more to explanations than models can tell](https://mlsanity.com/paper/2108.10363/) (2021)
- [Why Don't You Do Something About It? Outlining Connections between AI Explanations and User Actions](https://mlsanity.com/paper/2305.06297/) (2023)
- [Towards Assurance of LLM Adversarial Robustness using Ontology-Driven Argumentation](https://mlsanity.com/paper/2410.07962/) (2024)
- [Leveraging Clinical Context for User-Centered Explainability: A Diabetes Use Case](https://mlsanity.com/paper/2107.02359/) (2021)
- [Onto-Epistemological Analysis of AI Explanations](https://mlsanity.com/paper/2510.02996/) (2025)

## Abstract (arXiv)

Explainability has been a goal for Artificial Intelligence (AI) systems since their conception, with the need for explainability growing as more complex AI models are increasingly used in critical, high-stakes settings such as healthcare. Explanations have often added to an AI system in a non-principled, post-hoc manner. With greater adoption of these systems and emphasis on user-centric explainability, there is a need for a structured representation that treats explainability as a primary consideration, mapping end user needs to specific explanation types and the system's AI capabilities. We design an explanation ontology to model both the role of explanations, accounting for the system and user attributes in the process, and the range of different literature-derived explanation types. We indicate how the ontology can support user requirements for explanations in the domain of healthcare. We evaluate our ontology with a set of competency questions geared towards a system designer who might use our ontology to decide which explanation types to include, given a combination of users' needs and a system's capabilities, both in system design settings and in real-time operations. Through the use of this ontology, system designers will be able to make informed choices on which explanations AI systems can and should provide.