# Outlining the Design Space of Explainable Intelligent Systems for Medical Diagnosis

arXiv: https://arxiv.org/abs/1902.06019 · HTML: https://mlsanity.com/paper/1902.06019/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2019.

## Built on by (most cited first, 8 of 8)

- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023)
- [Explainable Artificial Intelligence (XAI) from a user perspective- A synthesis of prior literature and problematizing avenues for future research](https://mlsanity.com/paper/2211.15343/) (2022)
- [Explainable Artificial Intelligence (XAI): An Engineering Perspective](https://mlsanity.com/paper/2101.03613/) (2021)
- [Explainable Artificial Intelligence for Smart City Application: A Secure and Trusted Platform](https://mlsanity.com/paper/2111.00601/) (2021)
- [Explainable Interface for Human-Autonomy Teaming: A Survey](https://mlsanity.com/paper/2405.02583/) (2024)
- [Explainable Machine Learning: The Importance of a System-Centric Perspective](https://mlsanity.com/paper/2302.02347/) (2023)
- [Towards a Transparent and Interpretable AI Model for Medical Image Classifications](https://mlsanity.com/paper/2509.16685/) (2025)

## Abstract (arXiv)

The adoption of intelligent systems creates opportunities as well as challenges for medical work. On the positive side, intelligent systems have the potential to compute complex data from patients and generate automated diagnosis recommendations for doctors. However, medical professionals often perceive such systems as black boxes and, therefore, feel concerned about relying on system generated results to make decisions. In this paper, we contribute to the ongoing discussion of explainable artificial intelligence (XAI) by exploring the concept of explanation from a human-centered perspective. We hypothesize that medical professionals would perceive a system as explainable if the system was designed to think and act like doctors. We report a preliminary interview study that collected six medical professionals' reflection of how they interact with data for diagnosis and treatment purposes. Our data reveals when and how doctors prioritize among various types of data as a central part of their diagnosis process. Based on these findings, we outline future directions regarding the design of XAI systems in the medical context.