mlsanityData to 7 Oct 2026

Learning When-to-Treat Policies

arXiv 1905.09751 ↗First posted 23 May 2019stat.ME stat.MLcited by 49

Not on the map. Why?

Abstract (arXiv)

Many applied decision-making problems have a dynamic component: The policymaker needs not only to choose whom to treat, but also when to start which treatment. For example, a medical doctor may choose between postponing treatment (watchful waiting) and prescribing one of several available treatments during the many visits from a patient. We develop an "advantage doubly robust" estimator for learning such dynamic treatment rules using observational data under the assumption of sequential ignorability. We prove welfare regret bounds that generalize results for doubly robust learning in the single-step setting, and show promising empirical performance in several different contexts. Our approach is practical for policy optimization, and does not need any structural (e.g., Markovian) assumptions.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 13; 2022: 14; 2023: 11; 2024: 6; 2025: 4.

49 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 275
  3. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  4. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  5. 2021Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximationcited by 69
  6. 2022Distributionally Robust Offline Reinforcement Learning with Linear Function Approximationcited by 31
  7. 2023DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learningcited by 27
  8. 2021Active Offline Policy Selectioncited by 26
  9. 2021Offline Neural Contextual Bandits: Pessimism, Optimization and Generalizationcited by 25
  10. 2021Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorabilitycited by 24
  11. 2022On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximationcited by 22
  12. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 21
37 more
  1. 2024AlignIQL: Policy Alignment in Implicit Q-Learning through Constrained Optimizationcited by 19
  2. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learningcited by 14
  3. 2022Testing Stationarity and Change Point Detection in Reinforcement Learningcited by 12
  4. 2022A Behavior Regularized Implicit Policy for Offline Reinforcement Learningcited by 11
  5. 2022Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizonscited by 10
  6. 2023On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learningcited by 8
  7. 2023Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewalcited by 8
  8. 2021Estimating the Long-Term Effects of Novel Treatmentscited by 7
  9. 2022Offline Policy Optimization with Eligible Actionscited by 7
  10. 2021Stateful Offline Contextual Policy Evaluation and Learningcited by 6
  11. 2021Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approachcited by 6
  12. 2023HOPE: Human-Centric Off-Policy Evaluation for E-Learning and Healthcarecited by 5
  13. 2025CaRT: Teaching LLM Agents to Know When They Know Enoughcited by 5
  14. 2022Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performancecited by 4
  15. 2022Learning Optimal Dynamic Treatment Regimes Using Causal Tree Methods in Medicinecited by 4
  16. 2023VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximationcited by 3
  17. 2024Functional Acceleration for Policy Mirror Descentcited by 3
  18. 2023Infinite Action Contextual Bandits with Reusable Data Exhaustcited by 2
  19. 2024Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learningcited by 2
  20. 2025STITCH-OPE: Trajectory Stitching with Guided Diffusion for Off-Policy Evaluationcited by 2
  21. 2021Calibrated Optimal Decision Making with Multiple Data Sources and Limited Outcomecited by 1
  22. 2021GEAR: On Optimal Decision Making with Auxiliary Datacited by 1
  23. 2022Interpretable Deep Causal Learning for Moderation Effectscited by 1
  24. 2023π2vec: Policy Representations with Successor Featurescited by 1
  25. 2023Limits of Actor-Critic Algorithms for Decision Tree Policies Learning in IBMDPscited by 1
  26. 2021Dynamic covariate balancing: estimating treatment effects over time with potential local projectionscited by 0
  27. 2021Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraintscited by 0
  28. 2022Game-Theoretic Algorithms for Conditional Moment Matchingcited by 0
  29. 2022Counterfactual Learning with Multioutput Deep Kernelscited by 0
  30. 2023Deep Spectral Q-learning with Application to Mobile Healthcited by 0
  31. 2023Asymptotic Inference for Multi-Stage Stationary Treatment Policy with Variable Selectioncited by 0
  32. 2023A Semiparametric Instrumented Difference-in-Differences Approach to Policy Learningcited by 0
  33. 2024Artificial Intelligence-based Decision Support Systems for Precision and Digital Healthcited by 0
  34. 2024Fitted Q-Iteration via Max-Plus-Linear Approximationcited by 0
  35. 2024Off-dynamics Conditional Diffusion Plannerscited by 0
  36. 2025Time After Time: Deep-Q Effect Estimation for Interventions on When and What to docited by 0
  37. 2025Annotation-Assisted Learning of Treatment Policies From Multimodal Electronic Health Recordscited by 0