Backward Bayesian Outcome Weighted Learning

A central objective of precision medicine is learning optimal dynamic treatment regimes (DTRs) from data. Classification-based methods, like outcome weighted learning (OWL) for single-stage and backward OWL (BOWL) for multi-stage problems, leverage machine learning to directly learn optimal DTRs. However, these methods lack a natural way to quantify uncertainty in treatment decisions at the individual level. In this paper, we extend Bayesian OWL, a Bayesian reformulation of OWL, to the multi-stage setting. We call this method backward Bayesian outcome weighted learning (BBOWL). Like BOWL, our method directly learns an optimal DTR via backward induction, and unlike existing methods, our approach propagates uncertainty backward through the DTR learning process and provides uncertainty quantification of individualized treatment recommendations. We present a theoretical justification of BBOWL and verify its performance via a simulation study.