mlsanityData to 7 Oct 2026

Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions

arXiv 2203.02605 ↗First posted 4 Mar 2022stat.ML cs.LG stat.APcited by 6

Not on the map. Why?

Abstract (arXiv)

In recent years, reinforcement learning (RL) has acquired a prominent position in health-related sequential decision-making problems, gaining traction as a valuable tool for delivering adaptive interventions (AIs). However, in part due to a poor synergy between the methodological and the applied communities, its real-life application is still limited and its potential is still to be realized. To address this gap, our work provides the first unified technical survey on RL methods, complemented with case studies, for constructing various types of AIs in healthcare. In particular, using the common methodological umbrella of RL, we bridge two seemingly different AI domains, dynamic treatment regimes and just-in-time adaptive interventions in mobile health, highlighting similarities and differences between them and discussing the implications of using RL. Open problems and considerations for future research directions are outlined. Finally, we leverage our experience in designing case studies in both areas to showcase the significant collaborative opportunities between statistical, RL, and healthcare researchers in advancing AIs.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 37 of its 335 references that are in the map. First posted on arXiv 4 Mar 2022.

Builds on

37 of 335 references.

References · 37

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2017A Tutorial on Thompson Samplingcited by 420
  3. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  4. 2013Finite-Time Analysis of Kernelised Contextual Banditscited by 177
  5. 2016Identifying Causal Effects With Proxy Variables of an Unmeasured Confoundercited by 138
  6. 2015Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challengescited by 113
  7. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  8. 2019Confidence Intervals for Policy Evaluation in Adaptive Experimentscited by 95
  9. 2017Deep Reinforcement Learning for Sepsis Treatmentcited by 71
  10. 2019Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activitycited by 70
  11. 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
  12. 2016Estimating Dynamic Treatment Regimes in Mobile Health Using V-learningcited by 65
25 more
  1. 2021Statistical Inference with M-Estimators on Adaptively Collected Datacited by 40
  2. 2021Proximal Reinforcement Learning: Efficient Off-Policy Evaluation in Partially Observed Markov Decision Processescited by 40
  3. 2016A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patientscited by 38
  4. 2015Residual Weighted Learning for Estimating Individualized Treatment Rulescited by 37
  5. 2018Balanced Linear Contextual Banditscited by 30
  6. 2019Recovering Banditscited by 28
  7. 2016Assessing Time-Varying Causal Effect Moderation in Mobile Healthcited by 27
  8. 2021Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via pT-Learningcited by 26
  9. 2021Causal Decision Making and Causal Effect Estimation Are Not the Same... and Why It Matterscited by 25
  10. 2020IntelligentPooling: Practical Thompson Sampling for mHealthcited by 24
  11. 2017Action Centered Contextual Banditscited by 23
  12. 2018Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Datacited by 23
  13. 2021Online Multi-Armed Bandits with Adaptive Inferencecited by 21
  14. 2014Thompson sampling with the online bootstrapcited by 19
  15. 2018Semiparametric Contextual Banditscited by 19
  16. 2017An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventionscited by 17
  17. 2015Structural Nested Models and G-estimation: The Partially Realized Promisecited by 15
  18. 2019Direct and indirect reinforcement learningcited by 11
  19. 2019Contextual Multi-armed Bandit Algorithm for Semiparametric Reward Modelcited by 10
  20. 2018Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventionscited by 9
  21. 2021Efficient Inference Without Trading-off Regret in Bandits: An Allocation Probability Test for Thompson Samplingcited by 7
  22. 2019Neural Contextual Bandits with UCB-based Explorationcited by 6
  23. 2020Latent-state models for precision medicinecited by 6
  24. 2016A Batch, Off-Policy, Actor-Critic Algorithm for Optimizing the Average Rewardcited by 5
  25. 2017Estimating Individualized Treatment Rules for Ordinal Treatmentscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 2.

6 papers.

  1. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directionscited by 5
  2. 2024Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimescited by 3
  3. 2026BFTS: Thompson Sampling with Bayesian Additive Regression Treescited by 1
  4. 2023A Meta-Learning Method for Estimation of Causal Excursion Effects to Assess Time-Varying Moderationcited by 0
  5. 2024Artificial Intelligence-based Decision Support Systems for Precision and Digital Healthcited by 0
  6. 2025Conformal bandits: bringing statistical validity and reward efficiency under weak arm separabilitycited by 0