# Interpretable Dynamic Treatment Regimes

arXiv: https://arxiv.org/abs/1606.01472 · HTML: https://mlsanity.com/paper/1606.01472/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2016.

## Built on by (most cited first, 17 of 17)

- [A Review of Off-Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2212.06355/) (2022)
- [Estimating and Improving Dynamic Treatment Regimes With a Time-Varying Instrumental Variable](https://mlsanity.com/paper/2104.07822/) (2021)
- [Testing Stationarity and Change Point Detection in Reinforcement Learning](https://mlsanity.com/paper/2203.01707/) (2022)
- [Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons](https://mlsanity.com/paper/2202.13163/) (2022)
- [Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach](https://mlsanity.com/paper/2112.04571/) (2021)
- [ESCADA: Efficient Safety and Context Aware Dose Allocation for Precision Medicine](https://mlsanity.com/paper/2111.13415/) (2021)
- [Efficient and robust transfer learning of optimal individualized treatment regimes with right-censored survival data](https://mlsanity.com/paper/2301.05491/) (2023)
- [A Multi-Agent Reinforcement Learning Framework for Off-Policy Evaluation in Two-sided Markets](https://mlsanity.com/paper/2202.10574/) (2022)
- [Learning Optimal Dynamic Treatment Regimes Using Causal Tree Methods in Medicine](https://mlsanity.com/paper/2204.07124/) (2022)
- [Robust Sample Weighting to Facilitate Individualized Treatment Rule Learning for a Target Population](https://mlsanity.com/paper/2105.00581/) (2021)
- [Constrained Classification and Policy Learning](https://mlsanity.com/paper/2106.12886/) (2021)
- [Interpretable Off-Policy Learning via Hyperbox Search](https://mlsanity.com/paper/2203.02473/) (2022)
- [GEAR: On Optimal Decision Making with Auxiliary Data](https://mlsanity.com/paper/2104.10573/) (2021)
- [Safe and Interpretable Estimation of Optimal Treatment Regimes](https://mlsanity.com/paper/2310.15333/) (2023)
- [Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraints](https://mlsanity.com/paper/2106.05031/) (2021)
- [A Penalized Shared-parameter Algorithm for Estimating Optimal Dynamic Treatment Regimens](https://mlsanity.com/paper/2107.07875/) (2021)
- [Deep Spectral Q-learning with Application to Mobile Health](https://mlsanity.com/paper/2301.00927/) (2023)

## Abstract (arXiv)

Precision medicine is currently a topic of great interest in clinical and intervention science. One way to formalize precision medicine is through a treatment regime, which is a sequence of decision rules, one per stage of clinical intervention, that map up-to-date patient information to a recommended treatment. An optimal treatment regime is defined as maximizing the mean of some cumulative clinical outcome if applied to a population of interest. It is well-known that even under simple generative models an optimal treatment regime can be a highly nonlinear function of patient information. Consequently, a focal point of recent methodological research has been the development of flexible models for estimating optimal treatment regimes. However, in many settings, estimation of an optimal treatment regime is an exploratory analysis intended to generate new hypotheses for subsequent research and not to directly dictate treatment to new patients. In such settings, an estimated treatment regime that is interpretable in a domain context may be of greater value than an unintelligible treatment regime built using "black-box" estimation methods. We propose an estimator of an optimal treatment regime composed of a sequence of decision rules, each expressible as a list of "if-then" statements that can be presented as either a paragraph or as a simple flowchart that is immediately interpretable to domain experts. The discreteness of these lists precludes smooth, i.e., gradient-based, methods of estimation and leads to non-standard asymptotics. Nevertheless, we provide a computationally efficient estimation algorithm, prove consistency of the proposed estimator, and derive rates of convergence. We illustrate the proposed methods using a series of simulation examples and application to data from a sequential clinical trial on bipolar disorder.