mlsanityData to 14 Oct 2026

The Evidence Ladder for Reinforcement Learning in Healthcare: From Retrospective Policies to Trusted Interventions

arXiv 2609.23374 ↗First posted 20 Sep 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for clinical treatment and surgery

Abstract (arXiv)

Reinforcement learning (RL) offers a natural language for healthcare decisions whose conse- quences unfold over time, yet most reported progress remains far from routine intervention. Ex- isting surveys organize the field by algorithm or clinical application. We instead review healthcare RL through an evidence ladder: problem formulation, retrospective identification, policy estima- tion, stress testing, prospective evaluation, and lifecycle monitoring. This view connects clinical treatment, patient engagement, and health-system operations while exposing a recurring gap: evi- dence that a policy scores well in a historical dataset is not evidence that it will improve care. We synthesize the assumptions and failure modes at each rung, identify what evidence can and can- not transfer across settings, and propose reporting practices for cumulative evaluation. Restless bandits are included as one special case, not as the organizing framework. The central lesson is that healthcare RL should be evaluated as an intervention embedded in a changing sociotechnical system, rather than only as an optimizer of a retrospective reward.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods. Builds on 44 of its 106 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

44 of 106 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI28 Aug
  2. 2024Optimizing Vital Sign Monitoring in Resource-Constrained Maternal Care: An RL-Based Restless Bandit Approach10 Oct

Other references · 42

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  6. 2017Constrained Policy Optimizationcited by 983
  7. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  8. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  9. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  10. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 554
  11. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  12. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
30 more
  1. 2019Reinforcement Learning in Healthcare: A Surveycited by 310
  2. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  3. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 286
  4. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 227
  5. 2019A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cyclecited by 193
  6. 2019Minimax Weight and Q-Function Learning for Off-Policy Evaluationcited by 150
  7. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 144
  8. 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
  9. 2019Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processescited by 119
  10. 2019Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Modelscited by 117
  11. 2015Learning to Diagnose with LSTM Recurrent Neural Networkscited by 99
  12. 2019Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activitycited by 70
  13. 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
  14. 2019Algorithms for Fairness in Sequential Decision Makingcited by 56
  15. 2017A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Unitscited by 50
  16. 2020Collapsing Bandits and Their Application to Public Health Interventionscited by 47
  17. 2019Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learningcited by 34
  18. 2020An Empirical Study of Representation Learning for Reinforcement Learning in Healthcarecited by 34
  19. 2024A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Healthcited by 24
  20. 2022Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelinescited by 22
  21. 2021Nuances in Margin Conditions Determine Gains in Active Learningcited by 9
  22. 2023Scalable Neural Network Kernelscited by 9
  23. 2023Towards a Pretrained Model for Restless Bandits via Multi-arm Generalizationcited by 9
  24. 2024Towards Foundation-model-based Multiagent System to Accelerate AI for Social Impactcited by 9
  25. 2021Restless and Uncertain: Robust Policies for Restless Bandits via Deep Multi-Agent Reinforcement Learningcited by 7
  26. 2023Efficient Graph Field Integrators Meet Point Cloudscited by 7
  27. 2023Balanced Off-Policy Evaluation for Personalized Pricingcited by 5
  28. 2024The Bandit Whisperer: Communication Learning for Restless Banditscited by 5
  29. 2022Implicit Two-Tower Policiescited by 3
  30. 2024Improving the Prediction of Individual Engagement in Recommendations Using Cognitive Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Exploring Reinforcement Learning for Fluid Transitions Between Clinical Mental Healthcare and Everyday Wellness Support
  2. 2026Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management
  3. 2024Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination
  4. 2025Causal Reinforcement Learning based Agent-Patient Interaction with Clinical Domain Knowledge
  5. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions
  6. 2025Test-Time Learning and Inference-Time Deliberation for Efficiency-First Offline Reinforcement Learning in Care Coordination and Population Health Management
  7. 2026Boosted Distributional Reinforcement Learning: Analysis and Healthcare Applications
  8. 2024Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes