# Statistical Tests for Replacing Human Decision Makers with Algorithms

arXiv: https://arxiv.org/abs/2306.11689 · HTML: https://mlsanity.com/paper/2306.11689/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 42 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Builds on (most cited first, 6 of 6)

- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Direct Uncertainty Prediction for Medical Second Opinions](https://mlsanity.com/paper/1807.01771/) (2018)
- [Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks](https://mlsanity.com/paper/1707.01836/) (2017)
- [Interpretable Classification Models for Recidivism Prediction](https://mlsanity.com/paper/1503.07810/) (2015)
- [A comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis](https://mlsanity.com/paper/1806.10698/) (2018)

## Built on by (most cited first, 1 of 1)

- [The Challenger: When Do New Data Sources Justify Switching Machine Learning Models?](https://mlsanity.com/paper/2512.18390/) (2025)

## Abstract (arXiv)

This paper proposes a statistical framework of using artificial intelligence to improve human decision making. The performance of each human decision maker is benchmarked against that of machine predictions. We replace the diagnoses made by a subset of the decision makers with the recommendation from the machine learning algorithm. We apply both a heuristic frequentist approach and a Bayesian posterior loss function approach to abnormal birth detection using a nationwide dataset of doctor diagnoses from prepregnancy checkups of reproductive age couples and pregnancy outcomes. We find that our algorithm on a test dataset results in a higher overall true positive rate and a lower false positive rate than the diagnoses made by doctors only.