# Towards an Explanation Space to Align Humans and Explainable-AI Teamwork

arXiv: https://arxiv.org/abs/2106.01503 · HTML: https://mlsanity.com/paper/2106.01503/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 82 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI](https://mlsanity.com/paper/1902.01876/) (2019)
- [Stakeholders in Explainable AI](https://mlsanity.com/paper/1810.00184/) (2018)
- [A systematic review and taxonomy of explanations in decision support and recommender systems](https://mlsanity.com/paper/2006.08672/) (2020)
- [When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures](https://mlsanity.com/paper/1909.03418/) (2019)
- [Principles of Explanation in Human-AI Systems](https://mlsanity.com/paper/2102.04972/) (2021)
- [Generating Counterfactual and Contrastive Explanations using SHAP](https://mlsanity.com/paper/1906.09293/) (2019)
- [Interpreting Classifiers through Attribute Interactions in Datasets](https://mlsanity.com/paper/1707.07576/) (2017)
- [A Work-Centered Approach for Cyber-Physical-Social System Design: Applications in Aerospace Industrial Inspection](https://mlsanity.com/paper/2101.05385/) (2021)
- [Extending System Performance Past the Boundaries of Technical Maturity: Human-Agent Teamwork Perspective for Industrial Inspection](https://mlsanity.com/paper/2102.03241/) (2021)

## Abstract (arXiv)

Providing meaningful and actionable explanations to end-users is a fundamental prerequisite for implementing explainable intelligent systems in the real world. Explainability is a situated interaction between a user and the AI system rather than being static design principles. The content of explanations is context-dependent and must be defined by evidence about the user and its context. This paper seeks to operationalize this concept by proposing a formative architecture that defines the explanation space from a user-inspired perspective. The architecture comprises five intertwined components to outline explanation requirements for a task: (1) the end-users mental models, (2) the end-users cognitive process, (3) the user interface, (4) the human-explainer agent, and the (5) agent process. We first define each component of the architecture. Then we present the Abstracted Explanation Space, a modeling tool that aggregates the architecture's components to support designers in systematically aligning explanations with the end-users work practices, needs, and goals. It guides the specifications of what needs to be explained (content - end-users mental model), why this explanation is necessary (context - end-users cognitive process), to delimit how to explain it (format - human-explainer agent and user interface), and when should the explanations be given. We then exemplify the tool's use in an ongoing case study in the aircraft maintenance domain. Finally, we discuss possible contributions of the tool, known limitations/areas for improvement, and future work to be done.