# Improving Health Professionals' Onboarding with AI and XAI for Trustworthy Human-AI Collaborative Decision Making

arXiv: https://arxiv.org/abs/2405.16424 · HTML: https://mlsanity.com/paper/2405.16424/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 73 references that are in the map.
- First posted on arXiv 26 May 2024.

## Builds on (most cited first, 25 of 25)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review](https://mlsanity.com/paper/2010.10596/) (2020)
- [Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI](https://mlsanity.com/paper/2010.07487/) (2020)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making](https://mlsanity.com/paper/1902.02960/) (2019)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [The relationship between trust in AI and trustworthy machine learning technologies](https://mlsanity.com/paper/1912.00782/) (2019)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Trustworthy AI](https://mlsanity.com/paper/2011.02272/) (2020)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes](https://mlsanity.com/paper/1904.09612/) (2019)
- ["The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning](https://mlsanity.com/paper/1911.08089/) (2019)
- [Embedding Comparator: Visualizing Differences in Global Structure and Local Neighborhoods via Small Multiples](https://mlsanity.com/paper/1912.04853/) (2019)
- [Efficient Saliency Maps for Explainable AI](https://mlsanity.com/paper/1911.11293/) (2019)
- [Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services](https://mlsanity.com/paper/2303.09743/) (2023)
- ["Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment](https://mlsanity.com/paper/2101.01524/) (2021)
- [Ethics-Based Auditing to Develop Trustworthy AI](https://mlsanity.com/paper/2105.00002/) (2021)

## Built on by (most cited first, 2 of 2)

- [Clinicians' Voice: Fundamental Considerations for XAI in Healthcare](https://mlsanity.com/paper/2411.04855/) (2024)
- [Interactive Example-based Explanations to Improve Health Professionals' Onboarding with AI for Human-AI Collaborative Decision Making](https://mlsanity.com/paper/2409.15814/) (2024)

## Abstract (arXiv)

With advanced AI/ML, there has been growing research on explainable AI (XAI) and studies on how humans interact with AI and XAI for effective human-AI collaborative decision-making. However, we still have a lack of understanding of how AI systems and XAI should be first presented to users without technical backgrounds. In this paper, we present the findings of semi-structured interviews with health professionals (n=12) and students (n=4) majoring in medicine and health to study how to improve onboarding with AI and XAI. For the interviews, we built upon human-AI interaction guidelines to create onboarding materials of an AI system for stroke rehabilitation assessment and AI explanations and introduce them to the participants. Our findings reveal that beyond presenting traditional performance metrics on AI, participants desired benchmark information, the practical benefits of AI, and interaction trials to better contextualize AI performance, and refine the objectives and performance of AI. Based on these findings, we highlight directions for improving onboarding with AI and XAI and human-AI collaborative decision-making.