# Towards Trustworthy Automatic Diagnosis Systems by Emulating Doctors' Reasoning with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2210.07198 · HTML: https://mlsanity.com/paper/2210.07198/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 61 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Noisy Networks for Exploration](https://mlsanity.com/paper/1706.10295/) (2017)
- [DDXPlus: A New Dataset For Automatic Medical Diagnosis](https://mlsanity.com/paper/2205.09148/) (2022)
- [End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis](https://mlsanity.com/paper/1901.10623/) (2019)
- [rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch](https://mlsanity.com/paper/1909.01500/) (2019)
- [Classification with Costly Features using Deep Reinforcement Learning](https://mlsanity.com/paper/1711.07364/) (2017)
- [Opportunistic Learning: Budgeted Cost-Sensitive Learning from Data Streams](https://mlsanity.com/paper/1901.00243/) (2019)
- [Diaformer: Automatic Diagnosis via Symptoms Sequence Generation](https://mlsanity.com/paper/2112.10433/) (2021)
- [Efficient Symptom Inquiring and Diagnosis via Adaptive Alignment of Reinforcement Learning and Classification](https://mlsanity.com/paper/2112.00733/) (2021)
- ["My nose is running.""Are you also coughing?": Building A Medical Diagnosis Agent with Interpretable Inquiry Logics](https://mlsanity.com/paper/2204.13953/) (2022)
- [Knowledge Grounded Conversational Symptom Detection with Graph Memory Networks](https://mlsanity.com/paper/2101.09773/) (2021)
- [A Bayesian Approach for Medical Inquiry and Disease Inference in Automated Differential Diagnosis](https://mlsanity.com/paper/2110.08393/) (2021)

## Built on by (most cited first, 5 of 5)

- [ProtoGate: Prototype-based Neural Networks with Global-to-local Feature Selection for Tabular Biomedical Data](https://mlsanity.com/paper/2306.12330/) (2023)
- [Interpretable Differential Diagnosis with Dual-Inference Large Language Models](https://mlsanity.com/paper/2407.07330/) (2024)
- [Leveraging Anatomical Constraints with Uncertainty for Pneumothorax Segmentation](https://mlsanity.com/paper/2311.15213/) (2023)
- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models](https://mlsanity.com/paper/2412.15748/) (2024)

## Abstract (arXiv)

The automation of the medical evidence acquisition and diagnosis process has recently attracted increasing attention in order to reduce the workload of doctors and democratize access to medical care. However, most works proposed in the machine learning literature focus solely on improving the prediction accuracy of a patient's pathology. We argue that this objective is insufficient to ensure doctors' acceptability of such systems. In their initial interaction with patients, doctors do not only focus on identifying the pathology a patient is suffering from; they instead generate a differential diagnosis (in the form of a short list of plausible diseases) because the medical evidence collected from patients is often insufficient to establish a final diagnosis. Moreover, doctors explicitly explore severe pathologies before potentially ruling them out from the differential, especially in acute care settings. Finally, for doctors to trust a system's recommendations, they need to understand how the gathered evidences led to the predicted diseases. In particular, interactions between a system and a patient need to emulate the reasoning of doctors. We therefore propose to model the evidence acquisition and automatic diagnosis tasks using a deep reinforcement learning framework that considers three essential aspects of a doctor's reasoning, namely generating a differential diagnosis using an exploration-confirmation approach while prioritizing severe pathologies. We propose metrics for evaluating interaction quality based on these three aspects. We show that our approach performs better than existing models while maintaining competitive pathology prediction accuracy.