# Limits of trust in medical AI

arXiv: https://arxiv.org/abs/2503.16692 · HTML: https://mlsanity.com/paper/2503.16692/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 36 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 3 of 3)

- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [Massively Multitask Networks for Drug Discovery](https://mlsanity.com/paper/1502.02072/) (2015)

## Built on by (most cited first, 7 of 7)

- [A moving target in AI-assisted decision-making: Dataset shift, model updating, and the problem of update opacity](https://mlsanity.com/paper/2504.05210/) (2025)
- [Diachronic and synchronic variation in the performance of adaptive machine learning systems: The ethical challenges](https://mlsanity.com/paper/2504.08861/) (2025)
- [Are clinicians ethically obligated to disclose their use of medical machine learning systems to patients?](https://mlsanity.com/paper/2504.01043/) (2025)
- [Federated learning, ethics, and the double black box problem in medical AI](https://mlsanity.com/paper/2504.20656/) (2025)
- [In defence of post-hoc explanations in medical AI](https://mlsanity.com/paper/2504.20741/) (2025)
- [Explainability of Algorithms](https://mlsanity.com/paper/2508.13529/) (2025)
- [Principles and Guidelines for Randomized Controlled Trials in AI Evaluation](https://mlsanity.com/paper/2605.02050/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI) is expected to revolutionize the practice of medicine. Recent advancements in the field of deep learning have demonstrated success in a variety of clinical tasks: detecting diabetic retinopathy from images, predicting hospital readmissions, aiding in the discovery of new drugs, etc. AI's progress in medicine, however, has led to concerns regarding the potential effects of this technology upon relationships of trust in clinical practice. In this paper, I will argue that there is merit to these concerns, since AI systems can be relied upon, and are capable of reliability, but cannot be trusted, and are not capable of trustworthiness. Insofar as patients are required to rely upon AI systems for their medical decision-making, there is potential for this to produce a deficit of trust in relationships in clinical practice.