# Reliable Off-Policy Learning for Dosage Combinations

arXiv: https://arxiv.org/abs/2305.19742 · HTML: https://mlsanity.com/paper/2305.19742/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 93 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 30 of 51)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1604.00923/) (2016)
- [On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems](https://mlsanity.com/paper/1906.00331/) (2019)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Learning Likelihoods with Conditional Normalizing Flows](https://mlsanity.com/paper/1912.00042/) (2019)
- [Doubly Robust Policy Evaluation and Optimization](https://mlsanity.com/paper/1503.02834/) (2015)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Causal Transformer for Estimating Counterfactual Outcomes](https://mlsanity.com/paper/2204.07258/) (2022)
- [The Blessings of Multiple Causes](https://mlsanity.com/paper/1805.06826/) (2018)
- [Learning Counterfactual Representations for Estimating Individual Dose-Response Curves](https://mlsanity.com/paper/1902.00981/) (2019)
- [Deep Generalized Method of Moments for Instrumental Variable Analysis](https://mlsanity.com/paper/1905.12495/) (2019)
- [Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks](https://mlsanity.com/paper/2002.12326/) (2020)
- [Learning Deep Features in Instrumental Variable Regression](https://mlsanity.com/paper/2010.07154/) (2020)
- [Offline Multi-Action Policy Learning: Generalization and Optimization](https://mlsanity.com/paper/1810.04778/) (2018)
- [VCNet and Functional Targeted Regularization For Learning Causal Effects of Continuous Treatments](https://mlsanity.com/paper/2103.07861/) (2021)
- [Identifying Causal-Effect Inference Failure with Uncertainty-Aware Models](https://mlsanity.com/paper/2007.00163/) (2020)
- [Conditional Density Estimation with Bayesian Normalising Flows](https://mlsanity.com/paper/1802.04908/) (2018)
- [Balanced Policy Evaluation and Learning](https://mlsanity.com/paper/1705.07384/) (2017)
- [Residual Unfairness in Fair Machine Learning from Prejudiced Data](https://mlsanity.com/paper/1806.02887/) (2018)
- [Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting](https://mlsanity.com/paper/1902.01520/) (2019)

## Built on by (most cited first, 15 of 15)

- [Causal machine learning for predicting treatment outcomes](https://mlsanity.com/paper/2410.08770/) (2024)
- [Foundation Models for Causal Inference via Prior-Data Fitted Networks](https://mlsanity.com/paper/2506.10914/) (2025)
- [Bayesian Neural Controlled Differential Equations for Treatment Effect Estimation](https://mlsanity.com/paper/2310.17463/) (2023)
- [Conformal Prediction for Causal Effects of Continuous Treatments](https://mlsanity.com/paper/2407.03094/) (2024)
- [A Neural Framework for Generalized Causal Sensitivity Analysis](https://mlsanity.com/paper/2311.16026/) (2023)
- [Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments](https://mlsanity.com/paper/2406.02464/) (2024)
- [Treatment Effect Estimation for Optimal Decision-Making](https://mlsanity.com/paper/2505.13092/) (2025)
- [Efficient and Sharp Off-Policy Learning under Unobserved Confounding](https://mlsanity.com/paper/2502.13022/) (2025)
- [Rank-Learner: Orthogonal Ranking of Treatment Effects](https://mlsanity.com/paper/2602.03517/) (2026)
- [Sources of Gain: Decomposing Performance in Conditional Average Dose Response Estimation](https://mlsanity.com/paper/2406.08206/) (2024)
- [Doubly Robust Fusion of Many Treatments for Policy Learning](https://mlsanity.com/paper/2505.08092/) (2025)
- [ExOSITO: Explainable Off-Policy Learning with Side Information for Intensive Care Unit Blood Test Orders](https://mlsanity.com/paper/2504.17277/) (2025)
- [MOSIC: Model-Agnostic Optimal Subgroup Identification with Multi-Constraint for Improved Reliability](https://mlsanity.com/paper/2504.20908/) (2025)
- [Barycentric Fused Gromov-Wasserstein Balancing for Causal Inference under Multiple Treatments](https://mlsanity.com/paper/2608.22024/) (2026)
- [Weighted Spline-Expanded Networks with Distributional Balancing for Continuous Treatment Effects](https://mlsanity.com/paper/2609.33740/) (2026)

## Abstract (arXiv)

Decision-making in personalized medicine such as cancer therapy or critical care must often make choices for dosage combinations, i.e., multiple continuous treatments. Existing work for this task has modeled the effect of multiple treatments independently, while estimating the joint effect has received little attention but comes with non-trivial challenges. In this paper, we propose a novel method for reliable off-policy learning for dosage combinations. Our method proceeds along three steps: (1) We develop a tailored neural network that estimates the individualized dose-response function while accounting for the joint effect of multiple dependent dosages. (2) We estimate the generalized propensity score using conditional normalizing flows in order to detect regions with limited overlap in the shared covariate-treatment space. (3) We present a gradient-based learning algorithm to find the optimal, individualized dosage combinations. Here, we ensure reliable estimation of the policy value by avoiding regions with limited overlap. We finally perform an extensive evaluation of our method to show its effectiveness. To the best of our knowledge, ours is the first work to provide a method for reliable off-policy learning for optimal dosage combinations.