# Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis

arXiv: https://arxiv.org/abs/2309.12368 · HTML: https://mlsanity.com/paper/2309.12368/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 112 references that are in the map.
- First posted on arXiv 17 Sep 2023.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [GRAM: Graph-based Attention Model for Healthcare Representation Learning](https://mlsanity.com/paper/1611.07012/) (2016)
- [Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks](https://mlsanity.com/paper/1706.05764/) (2017)
- [Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making](https://mlsanity.com/paper/1902.02960/) (2019)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes](https://mlsanity.com/paper/1904.09612/) (2019)
- ["The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning](https://mlsanity.com/paper/1911.08089/) (2019)
- [Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care](https://mlsanity.com/paper/2302.00096/) (2023)
- [Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens](https://mlsanity.com/paper/2102.00593/) (2021)
- [Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging](https://mlsanity.com/paper/2205.09696/) (2022)
- ["Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment](https://mlsanity.com/paper/2101.01524/) (2021)
- [GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization](https://mlsanity.com/paper/2211.02733/) (2022)
- [Measuring Patient Similarities via a Deep Architecture with Medical Concept Embedding](https://mlsanity.com/paper/1902.03376/) (2019)
- [Deconfounding Actor-Critic Network with Policy Adaptation for Dynamic Treatment Regimes](https://mlsanity.com/paper/2205.09852/) (2022)
- [CarePre: An Intelligent Clinical Decision Assistance System](https://mlsanity.com/paper/1811.02218/) (2018)
- [Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering](https://mlsanity.com/paper/2206.12866/) (2022)
- [KnowledgeShovel: An AI-in-the-Loop Document Annotation System for Scientific Knowledge Base Construction](https://mlsanity.com/paper/2210.02830/) (2022)

## Built on by (most cited first, 30 of 30)

- [How Far Are We From AGI: Are LLMs All We Need?](https://mlsanity.com/paper/2405.10313/) (2024)
- [Mutual Theory of Mind in Human-AI Collaboration: An Empirical Study with LLM-driven AI Agents in a Real-time Shared Workspace Task](https://mlsanity.com/paper/2409.08811/) (2024)
- [Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges](https://mlsanity.com/paper/2507.19672/) (2025)
- [Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making](https://mlsanity.com/paper/2401.07058/) (2024)
- [More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineering](https://mlsanity.com/paper/2311.09782/) (2023)
- [AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support](https://mlsanity.com/paper/2504.06771/) (2025)
- [Position: Foundation Agents as the Paradigm Shift for Decision Making](https://mlsanity.com/paper/2405.17009/) (2024)
- [Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Making](https://mlsanity.com/paper/2503.18419/) (2025)
- [ClinicRealm: Re-evaluating Large Language Models with Conventional Machine Learning for Non-Generative Clinical Prediction Tasks](https://mlsanity.com/paper/2407.18525/) (2024)
- [Learning under Temporal Label Noise](https://mlsanity.com/paper/2402.04398/) (2024)
- [Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLM](https://mlsanity.com/paper/2410.14879/) (2024)
- [Unraveling Human-AI Teaming: A Review and Outlook](https://mlsanity.com/paper/2504.05755/) (2025)
- [Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration](https://mlsanity.com/paper/2509.18008/) (2025)
- [Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process](https://mlsanity.com/paper/2406.08959/) (2024)
- [A Survey on Human-Centered Evaluation of Explainable AI Methods in Clinical Decision Support Systems](https://mlsanity.com/paper/2502.09849/) (2025)
- [Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest](https://mlsanity.com/paper/2504.04243/) (2025)
- [EMRModel: A Large Language Model for Extracting Medical Consultation Dialogues into Structured Medical Records](https://mlsanity.com/paper/2504.16448/) (2025)
- [Augmenting Clinical Decision-Making with an Interactive and Interpretable AI Copilot: A Real-World User Study with Clinicians in Nephrology and Obstetrics](https://mlsanity.com/paper/2602.00726/) (2026)
- [AI-Driven Agents with Prompts Designed for High Agreeableness Increase the Likelihood of Being Mistaken for a Human in the Turing Test](https://mlsanity.com/paper/2411.13749/) (2024)
- ["It Felt Like I Was Left in the Dark": Exploring Information Needs and Design Opportunities for Family Caregivers of Older Adult Patients in Critical Care Settings](https://mlsanity.com/paper/2502.05115/) (2025)
- [Explanation User Interfaces: A Systematic Literature Review](https://mlsanity.com/paper/2505.20085/) (2025)
- [Balancing Efficiency and Empathy: Healthcare Providers' Perspectives on AI-Supported Workflows for Serious Illness Conversations in the Emergency Department](https://mlsanity.com/paper/2506.00241/) (2025)
- [Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks](https://mlsanity.com/paper/2507.10208/) (2025)
- [Early Risk Prediction with Temporally and Contextually Grounded Clinical Language Processing](https://mlsanity.com/paper/2511.22038/) (2025)
- [Intelligent Reasoning Cues: A Framework and Case Study of the Roles of AI Information in Complex Decisions](https://mlsanity.com/paper/2602.00259/) (2026)
- [Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making](https://mlsanity.com/paper/2602.11924/) (2026)
- [Within the MDT Room: Situated in Multidisciplinary Team-Grounded Agent Debate for Clinical Diagnosis](https://mlsanity.com/paper/2603.28393/) (2026)
- [From Clinical Intent to Clinical Model: Autonomous Coding-Agents for Clinician-driven AI Development](https://mlsanity.com/paper/2604.17110/) (2026)
- [PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI](https://mlsanity.com/paper/2605.05682/) (2026)
- [Co-Annotator: Expert-Distilled ViT and VLM for Visual and Documentation Guidance in Age-Related Macular Degeneration](https://mlsanity.com/paper/2608.30352/) (2026)

## Abstract (arXiv)

Today's AI systems for medical decision support often succeed on benchmark datasets in research papers but fail in real-world deployment. This work focuses on the decision making of sepsis, an acute life-threatening systematic infection that requires an early diagnosis with high uncertainty from the clinician. Our aim is to explore the design requirements for AI systems that can support clinical experts in making better decisions for the early diagnosis of sepsis. The study begins with a formative study investigating why clinical experts abandon an existing AI-powered Sepsis predictive module in their electrical health record (EHR) system. We argue that a human-centered AI system needs to support human experts in the intermediate stages of a medical decision-making process (e.g., generating hypotheses or gathering data), instead of focusing only on the final decision. Therefore, we build SepsisLab based on a state-of-the-art AI algorithm and extend it to predict the future projection of sepsis development, visualize the prediction uncertainty, and propose actionable suggestions (i.e., which additional laboratory tests can be collected) to reduce such uncertainty. Through heuristic evaluation with six clinicians using our prototype system, we demonstrate that SepsisLab enables a promising human-AI collaboration paradigm for the future of AI-assisted sepsis diagnosis and other high-stakes medical decision making.