mlsanityData to 7 Oct 2026

Clinicians' Voice: Fundamental Considerations for XAI in Healthcare

arXiv 2411.04855 ↗First posted 7 Nov 2024cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Explainable AI (XAI) holds the promise of advancing the implementation and adoption of AI-based tools in practice, especially in high-stakes environments like healthcare. However, most of the current research lacks input from end users, and therefore their practical value is limited. To address this, we conducted semi-structured interviews with clinicians to discuss their thoughts, hopes, and concerns. Clinicians from our sample generally think positively about developing AI-based tools for clinical practice, but they have concerns about how these will fit into their workflow and how it will impact clinician-patient relations. We further identify training of clinicians on AI as a crucial factor for the success of AI in healthcare and highlight aspects clinicians are looking for in (X)AI-based tools. In contrast to other studies, we take on a holistic and exploratory perspective to identify general requirements for (X)AI products for healthcare before moving on to testing specific tools.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 73 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

27 of 73 references.

References · 27

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  3. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  4. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  5. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  6. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 612
  7. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  8. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 403
  9. 2018Actionable Recourse in Linear Classificationcited by 327
  10. 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
  11. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  12. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
15 more
  1. 2019Model-Agnostic Counterfactual Explanations for Consequential Decisionscited by 204
  2. 2019Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifierscited by 143
  3. 2019What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Usecited by 134
  4. 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
  5. 2019Efficient Search for Diverse Coherent Explanationscited by 111
  6. 2020The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategiescited by 91
  7. 2019On the Existence of Simpler Machine Learning Modelscited by 87
  8. 2019Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processescited by 53
  9. 2023Finding Regions of Counterfactual Explanations via Robust Optimizationcited by 19
  10. 2024Explainable AI improves task performance in human-AI collaborationcited by 11
  11. 2022Counterfactual Explanations Using Optimization With Constraint Learningcited by 8
  12. 2021Rule Generation for Classification: Scalability, Interpretability, and Fairnesscited by 4
  13. 2022Explainability's Gain is Optimality's Loss? -- How Explanations Bias Decision-makingcited by 4
  14. 2023Semantic match: Debugging feature attribution methods in XAI for healthcarecited by 2
  15. 2024Improving Health Professionals' Onboarding with AI and XAI for Trustworthy Human-AI Collaborative Decision Makingcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Artificial Intelligence Should Genuinely Support Clinical Reasoning and Decision Making To Bridge the Translational Gapcited by 13
  2. 2025From Prototypes to Sparse ECG Explanations: SHAP-Driven Counterfactuals for Multivariate Time-Series Multi-class Classificationcited by 3
  3. 2025Visual-Conversational Interface for Evidence-Based Explanation of Diabetes Risk Predictioncited by 2