mlsanityData to 7 Oct 2026

Estimating Dynamic Treatment Regimes in Mobile Health Using V-learning

arXiv 1611.03531 ↗First posted 10 Nov 2016stat.MLcited by 65

Not on the map. Why?

Abstract (arXiv)

The vision for precision medicine is to use individual patient characteristics to inform a personalized treatment plan that leads to the best healthcare possible for each patient. Mobile technologies have an important role to play in this vision as they offer a means to monitor a patient's health status in real-time and subsequently to deliver interventions if, when, and in the dose that they are needed. Dynamic treatment regimes formalize individualized treatment plans as sequences of decision rules, one per stage of clinical intervention, that map current patient information to a recommended treatment. However, existing methods for estimating optimal dynamic treatment regimes are designed for a small number of fixed decision points occurring on a coarse time-scale. We propose a new reinforcement learning method for estimating an optimal treatment regime that is applicable to data collected using mobile technologies in an outpatient setting. The proposed method accommodates an indefinite time horizon and minute-by-minute decision making that are common in mobile health applications. We show the proposed estimators are consistent and asymptotically normal under mild conditions. The proposed methods are applied to estimate an optimal dynamic treatment regime for controlling blood glucose levels in patients with type 1 diabetes.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 8; 2022: 20; 2023: 15; 2024: 8; 2025: 8; 2026: 5.

65 papers.

Sort:

By area:

  1. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
  2. 2022Off-Policy Confidence Interval Estimation with Confounded Markov Decision Processcited by 31
  3. 2021A Statistical Analysis of Polyak-Ruppert Averaged Q-learningcited by 29
  4. 2021Online Bootstrap Inference For Policy Evaluation in Reinforcement Learningcited by 26
  5. 2021Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via pT-Learningcited by 26
  6. 2021Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorabilitycited by 24
  7. 2022An Instrumental Variable Approach to Confounded Off-Policy Evaluationcited by 20
  8. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statisticcited by 19
  9. 2021Projected State-action Balancing Weights for Offline Reinforcement Learningcited by 16
  10. 2022Conformal Off-policy Predictioncited by 14
  11. 2022Federated Offline Reinforcement Learningcited by 13
  12. 2022Testing Stationarity and Change Point Detection in Reinforcement Learningcited by 12
38 more
  1. 2022Redefining Counterfactual Explanations for Reinforcement Learning: Overview, Challenges and Opportunitiescited by 12
  2. 2022Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizonscited by 10
  3. 2020Efficient Evaluation of Natural Stochastic Policies in Offline Reinforcement Learningcited by 8
  4. 2022Policy Evaluation for Temporal and/or Spatial Dependent Experimentscited by 8
  5. 2022Reinforcement Learning with Heterogeneous Data: Estimation and Inferencecited by 7
  6. 2023Value Enhancement of Reinforcement Learning via Efficient and Robust Trust Region Optimizationcited by 7
  7. 2023Quasi-optimal Reinforcement Learning with Continuous Actionscited by 7
  8. 2023Sequential Knockoffs for Variable Selection in Reinforcement Learningcited by 7
  9. 2023Off-policy Evaluation in Doubly Inhomogeneous Environmentscited by 7
  10. 2023Robust Offline Reinforcement learning with Heavy-Tailed Rewardscited by 7
  11. 2021Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approachcited by 6
  12. 2022Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventionscited by 6
  13. 2023Distributional Shift-Aware Off-Policy Interval Estimation: A Unified Error Quantification Frameworkcited by 6
  14. 2024Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experimentscited by 6
  15. 2024Combining Experimental and Historical Data for Policy Evaluationcited by 6
  16. 2022Quantile Off-Policy Evaluation via Deep Conditional Generative Learningcited by 5
  17. 2023Revisiting Bellman Errors for Offline Model Selectioncited by 5
  18. 2023Bi-Level Offline Policy Optimization with Limited Explorationcited by 5
  19. 2024Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexaminationcited by 5
  20. 2025Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approachcited by 5
  21. 2022A Multi-Agent Reinforcement Learning Framework for Off-Policy Evaluation in Two-sided Marketscited by 4
  22. 2022Learning Optimal Dynamic Treatment Regimes Using Causal Tree Methods in Medicinecited by 4
  23. 2021Adaptive Sequential Design for a Single Time-Seriescited by 3
  24. 2021Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Surveycited by 3
  25. 2022Transfer Q-learningcited by 3
  26. 2022Doubly Inhomogeneous Reinforcement Learningcited by 3
  27. 2024Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimescited by 3
  28. 2025Reinforcement Learning with Continuous Actions Under Unmeasured Confoundingcited by 3
  29. 2025Reinforcement Learning for Individual Optimal Policy from Heterogeneous Datacited by 3
  30. 2022Targeted Optimal Treatment Regime Learning Using Summary Statisticscited by 2
  31. 2022Relative Sparsity for Medical Decision Problemscited by 2
  32. 2023A Reinforcement Learning Framework for Dynamic Mediation Analysiscited by 2
  33. 2024Off-policy Evaluation with Deeply-abstracted Statescited by 2
  34. 2024Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferencescited by 2
  35. 2025Statistical Inference in Reinforcement Learning: A Selective Surveycited by 2
  36. 2025Off-Policy Evaluation Under Nonignorable Missing Datacited by 2
  37. 2025A Two-armed Bandit Framework for A/B Testingcited by 2
  38. 2022IGN : Implicit Generative Networkscited by 1

50 most cited of 65 All 65 (JSON)