# Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes

arXiv: https://arxiv.org/abs/1904.09612 · HTML: https://mlsanity.com/paper/1904.09612/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2019.

## Built on by (most cited first, 30 of 53)

- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- [Fair Conformal Predictors for Applications in Medical Imaging](https://mlsanity.com/paper/2109.04392/) (2021)
- [Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making](https://mlsanity.com/paper/2101.05303/) (2021)
- [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022)
- [Role of Human-AI Interaction in Selective Prediction](https://mlsanity.com/paper/2112.06751/) (2021)
- [Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI](https://mlsanity.com/paper/2302.00799/) (2023)
- [Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis Diagnosis](https://mlsanity.com/paper/2309.12368/) (2023)
- [How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study](https://mlsanity.com/paper/2101.06098/) (2021)
- ["Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment](https://mlsanity.com/paper/2101.01524/) (2021)
- [Designing for human-AI complementarity in K-12 education](https://mlsanity.com/paper/2104.01266/) (2021)
- [Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables](https://mlsanity.com/paper/2207.13834/) (2022)
- [Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making](https://mlsanity.com/paper/2308.04375/) (2023)
- [Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making](https://mlsanity.com/paper/2204.05030/) (2022)
- [The challenge of uncertainty quantification of large language models in medicine](https://mlsanity.com/paper/2504.05278/) (2025)
- [Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models](https://mlsanity.com/paper/2102.00036/) (2021)
- [VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models](https://mlsanity.com/paper/2108.02550/) (2021)
- [Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system](https://mlsanity.com/paper/2307.15793/) (2023)
- [Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making](https://mlsanity.com/paper/2302.08807/) (2023)
- [An Uncommon Task: Participatory Design in Legal AI](https://mlsanity.com/paper/2203.06246/) (2022)
- [CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative Models](https://mlsanity.com/paper/2302.05678/) (2023)
- [AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development](https://mlsanity.com/paper/2502.18682/) (2025)
- [HypER: Literature-grounded Hypothesis Generation and Distillation with Provenance](https://mlsanity.com/paper/2506.12937/) (2025)
- [Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare](https://mlsanity.com/paper/2211.16444/) (2022)
- [Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI](https://mlsanity.com/paper/2308.07213/) (2023)
- [Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and Use](https://mlsanity.com/paper/2405.12458/) (2024)
- [Contextualized Machine Learning](https://mlsanity.com/paper/2310.11340/) (2023)
- [Clinical Evidence Engine: Proof-of-Concept For A Clinical-Domain-Agnostic Decision Support Infrastructure](https://mlsanity.com/paper/2111.00621/) (2021)
- [Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process](https://mlsanity.com/paper/2406.08959/) (2024)
- [Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers](https://mlsanity.com/paper/2410.22937/) (2024)
- [Clinicians' Voice: Fundamental Considerations for XAI in Healthcare](https://mlsanity.com/paper/2411.04855/) (2024)

## Abstract (arXiv)

Clinical decision support tools (DST) promise improved healthcare outcomes by offering data-driven insights. While effective in lab settings, almost all DSTs have failed in practice. Empirical research diagnosed poor contextual fit as the cause. This paper describes the design and field evaluation of a radically new form of DST. It automatically generates slides for clinicians' decision meetings with subtly embedded machine prognostics. This design took inspiration from the notion of "Unremarkable Computing", that by augmenting the users' routines technology/AI can have significant importance for the users yet remain unobtrusive. Our field evaluation suggests clinicians are more likely to encounter and embrace such a DST. Drawing on their responses, we discuss the importance and intricacies of finding the right level of unremarkableness in DST design, and share lessons learned in prototyping critical AI systems as a situated experience.