# Deep Reinforcement Learning for Cost-Effective Medical Diagnosis

arXiv: https://arxiv.org/abs/2302.10261 · HTML: https://mlsanity.com/paper/2302.10261/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 58 references that are in the map.
- First posted on arXiv 20 Feb 2023.

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Variational Policy Gradient Method for Reinforcement Learning with General Utilities](https://mlsanity.com/paper/2007.02151/) (2020)
- [Active Feature Acquisition with Generative Surrogate Models](https://mlsanity.com/paper/2010.02433/) (2020)
- [Classification with Costly Features using Deep Reinforcement Learning](https://mlsanity.com/paper/1711.07364/) (2017)
- [Reinforcement Learning with Efficient Active Feature Acquisition](https://mlsanity.com/paper/2011.00825/) (2020)
- [Sequential Cost-Sensitive Feature Acquisition](https://mlsanity.com/paper/1607.03691/) (2016)
- [EMFlow: Data Imputation in Latent Space via EM and Deep Flow Models](https://mlsanity.com/paper/2106.04804/) (2021)
- [Early Prediction of Mortality in Critical Care Setting in Sepsis Patients Using Structured Features and Unstructured Clinical Notes](https://mlsanity.com/paper/2112.01230/) (2021)

## Built on by (most cited first, 21 of 21)

- [Language Agents for Hypothesis-driven Clinical Decision Making with Reinforcement Learning](https://mlsanity.com/paper/2506.13474/) (2025)
- [ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistance](https://mlsanity.com/paper/2402.13448/) (2024)
- [Timely Clinical Diagnosis through Active Test Selection](https://mlsanity.com/paper/2510.18988/) (2025)
- [Deep Reinforcement Learning for Personalized Diagnostic Decision Pathways Using Electronic Health Records: A Comparative Study on Anemia and Systemic Lupus Erythematosus](https://mlsanity.com/paper/2404.05913/) (2024)
- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [Limited Query Graph Connectivity Test](https://mlsanity.com/paper/2302.13036/) (2023)
- [Learning Computational Efficient Bots with Costly Features](https://mlsanity.com/paper/2308.09629/) (2023)
- [PRECISE-AS: Personalized Reinforcement Learning for Efficient Point-of-Care Echocardiography in Aortic Stenosis Diagnosis](https://mlsanity.com/paper/2509.02898/) (2025)
- [Uncertainty-Guided Latent Diagnostic Trajectory Learning for Sequential Clinical Diagnosis](https://mlsanity.com/paper/2604.05116/) (2026)
- [Dynamic feature selection in medical predictive monitoring by reinforcement learning](https://mlsanity.com/paper/2405.19729/) (2024)
- [Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia](https://mlsanity.com/paper/2409.15377/) (2024)
- [Learning to Stop: Reinforcement Learning for Efficient Patient-Level Echocardiographic Classification](https://mlsanity.com/paper/2509.19694/) (2025)
- [Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agents](https://mlsanity.com/paper/2609.09678/) (2026)
- [Extracting Diagnosis Pathways from Electronic Health Records Using Deep Reinforcement Learning](https://mlsanity.com/paper/2305.06295/) (2023)
- [Optimizing Gene-Based Testing for Antibiotic Resistance Prediction](https://mlsanity.com/paper/2502.14919/) (2025)
- [Adaptive Wizard for Removing Cross-Tier Misconfigurations in Active Directory](https://mlsanity.com/paper/2505.01028/) (2025)
- [Online Learning of Optimal Sequential Testing Policies](https://mlsanity.com/paper/2509.03707/) (2025)
- [Optimizing Point-of-Care Ultrasound Video Acquisition for Probabilistic Multi-Task Heart Failure Detection](https://mlsanity.com/paper/2602.13658/) (2026)
- [Practical Graph Optimisation and AI-Driven Models for Active Directory Security Hardening](https://mlsanity.com/paper/2607.22009/) (2026)
- [Primal--Dual Alternating Neural Learning for Timely Classification with Performance Guarantees](https://mlsanity.com/paper/2608.23480/) (2026)
- [SCOPE-AD: Sequential cost-aware ordinal-belief planning with energy-based models for diagnostic agents](https://mlsanity.com/paper/2610.01278/) (2026)

## Abstract (arXiv)

Dynamic diagnosis is desirable when medical tests are costly or time-consuming. In this work, we use reinforcement learning (RL) to find a dynamic policy that selects lab test panels sequentially based on previous observations, ensuring accurate testing at a low cost. Clinical diagnostic data are often highly imbalanced; therefore, we aim to maximize the F1 score instead of the error rate. However, optimizing the non-concave F1 score is not a classic RL problem, thus invalidates standard RL methods. To remedy this issue, we develop a reward shaping approach, leveraging properties of the F1 score and duality of policy optimization, to provably find the set of all Pareto-optimal policies for budget-constrained F1 score maximization. To handle the combinatorially complex state space, we propose a Semi-Model-based Deep Diagnosis Policy Optimization (SM-DDPO) framework that is compatible with end-to-end training and online learning. SM-DDPO is tested on diverse clinical tasks: ferritin abnormality detection, sepsis mortality prediction, and acute kidney injury diagnosis. Experiments with real-world data validate that SM-DDPO trains efficiently and identifies all Pareto-front solutions. Across all tasks, SM-DDPO is able to achieve state-of-the-art diagnosis accuracy (in some cases higher than conventional methods) with up to 85% reduction in testing cost. The code is available at [https://github.com/Zheng321/Deep-Reinforcement-Learning-for-Cost-Effective-Medical-Diagnosis].