# Explainable AI for Clinical Outcome Prediction: A Survey of Clinician Perceptions and Preferences

arXiv: https://arxiv.org/abs/2502.20478 · HTML: https://mlsanity.com/paper/2502.20478/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies](https://mlsanity.com/paper/2007.15911/) (2020)
- [UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus](https://mlsanity.com/paper/2010.10391/) (2020)
- [How Different Groups Prioritize Ethical Values for Responsible AI](https://mlsanity.com/paper/2205.07722/) (2022)
- [Automated Lay Language Summarization of Biomedical Scientific Reviews](https://mlsanity.com/paper/2012.12573/) (2020)
- [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://mlsanity.com/paper/2102.04110/) (2021)
- [Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review](https://mlsanity.com/paper/1908.05780/) (2019)
- [Explainable AI for clinical and remote health applications: a survey on tabular and time series data](https://mlsanity.com/paper/2209.06528/) (2022)
- [Literature-Augmented Clinical Outcome Prediction](https://mlsanity.com/paper/2111.08374/) (2021)
- [APPLS: Evaluating Evaluation Metrics for Plain Language Summarization](https://mlsanity.com/paper/2305.14341/) (2023)
- [Improving Hospital Mortality Prediction with Medical Named Entities and Multimodal Learning](https://mlsanity.com/paper/1811.12276/) (2018)
- [Can LLMs like GPT-4 outperform traditional AI tools in dementia diagnosis? Maybe, but not today](https://mlsanity.com/paper/2306.01499/) (2023)
- [Explainable Causal Analysis of Mental Health on Social Media Data](https://mlsanity.com/paper/2210.08430/) (2022)
- [HiTZ@Antidote: Argumentation-driven Explainable Artificial Intelligence for Digital Medicine](https://mlsanity.com/paper/2306.06029/) (2023)
- [Attention Visualizer Package: Revealing Word Importance for Deeper Insight into Encoder-Only Transformer Models](https://mlsanity.com/paper/2308.14850/) (2023)
- [Constructing Ontology-Based Cancer Treatment Decision Support System with Case-Based Reasoning](https://mlsanity.com/paper/1812.01891/) (2018)

## Built on by (most cited first, 4 of 4)

- [A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior](https://mlsanity.com/paper/2602.02639/) (2026)
- [Cost-Aware Prediction (CAP): An LLM-Enhanced Machine Learning Pipeline and Decision Support System for Heart Failure Mortality Prediction](https://mlsanity.com/paper/2511.15357/) (2025)
- [TRIAGE: Dialectical Reasoning for Explainable Risk Prediction on Irregularly Sampled Medical Time Series with LLMs](https://mlsanity.com/paper/2606.09030/) (2026)
- [Addressing the Selection Problem in Explainable AI](https://mlsanity.com/paper/2608.22356/) (2026)

## Abstract (arXiv)

Explainable AI (XAI) techniques are necessary to help clinicians make sense of AI predictions and integrate predictions into their decision-making workflow. In this work, we conduct a survey study to understand clinician preference among different XAI techniques when they are used to interpret model predictions over text-based EHR data. We implement four XAI techniques (LIME, Attention-based span highlights, exemplar patient retrieval, and free-text rationales generated by LLMs) on an outcome prediction model that uses ICU admission notes to predict a patient's likelihood of experiencing in-hospital mortality. Using these XAI implementations, we design and conduct a survey study of 32 practicing clinicians, collecting their feedback and preferences on the four techniques. We synthesize our findings into a set of recommendations describing when each of the XAI techniques may be more appropriate, their potential limitations, as well as recommendations for improvement.