mlsanityData to 13 Oct 2026

Reliable Off-Policy Learning for Dosage Combinations

arXiv 2305.19742 ↗First posted 31 May 2023cs.LG cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Decision-making in personalized medicine such as cancer therapy or critical care must often make choices for dosage combinations, i.e., multiple continuous treatments. Existing work for this task has modeled the effect of multiple treatments independently, while estimating the joint effect has received little attention but comes with non-trivial challenges. In this paper, we propose a novel method for reliable off-policy learning for dosage combinations. Our method proceeds along three steps: (1) We develop a tailored neural network that estimates the individualized dose-response function while accounting for the joint effect of multiple dependent dosages. (2) We estimate the generalized propensity score using conditional normalizing flows in order to detect regions with limited overlap in the shared covariate-treatment space. (3) We present a gradient-based learning algorithm to find the optimal, individualized dosage combinations. Here, we ensure reliable estimation of the policy value by avoiding regions with limited overlap. We finally perform an extensive evaluation of our method to show its effectiveness. To the best of our knowledge, ours is the first work to provide a method for reliable off-policy learning for optimal dosage combinations.

Key facts. Cited by 15 papers in the map (data through 13 Oct 2026). Builds on 51 of its 93 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

51 of 93 references.

References · 51

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2015Variational Inference with Normalizing Flowscited by 2,163
  3. 2016Density estimation using Real NVPcited by 2,033
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,782
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,231
  6. 2014NICE: Non-linear Independent Components Estimationcited by 1,054
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2019Neural Spline Flowscited by 590
  9. 2017Quasi-Oracle Estimation of Heterogeneous Treatment Effectscited by 330
  10. 2016Learning Representations for Counterfactual Inferencecited by 327
  11. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  12. 2019Adapting Neural Networks for the Estimation of Treatment Effectscited by 285
38 more
  1. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 266
  2. 2018Neural Autoregressive Flowscited by 207
  3. 2019Learning Likelihoods with Conditional Normalizing Flowscited by 171
  4. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  5. 2019Orthogonal Statistical Learningcited by 150
  6. 2022Causal Transformer for Estimating Counterfactual Outcomescited by 122
  7. 2018The Blessings of Multiple Causescited by 103
  8. 2019Learning Counterfactual Representations for Estimating Individual Dose-Response Curvescited by 90
  9. 2019Deep Generalized Method of Moments for Instrumental Variable Analysiscited by 88
  10. 2020Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networkscited by 78
  11. 2020Learning Deep Features in Instrumental Variable Regressioncited by 65
  12. 2018Offline Multi-Action Policy Learning: Generalization and Optimizationcited by 64
  13. 2021VCNet and Functional Targeted Regularization For Learning Causal Effects of Continuous Treatmentscited by 60
  14. 2020Identifying Causal-Effect Inference Failure with Uncertainty-Aware Modelscited by 57
  15. 2018Conditional Density Estimation with Bayesian Normalising Flowscited by 54
  16. 2017Balanced Policy Evaluation and Learningcited by 51
  17. 2018Residual Unfairness in Fair Machine Learning from Prejudiced Datacited by 49
  18. 2019Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adaptingcited by 42
  19. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
  20. 2016Generalized Optimal Matching Methods for Causal Inferencecited by 39
  21. 2020Invertible Generative Modeling using Linear Rational Splinescited by 39
  22. 2021Causal Effect Inference for Structured Treatmentscited by 38
  23. 2022Estimating average causal effects from patient trajectoriescited by 29
  24. 2019Semi-Parametric Efficient Policy Learning with Continuous Actionscited by 26
  25. 2021SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Datacited by 25
  26. 2019More Efficient Policy Learning via Optimal Retargetingcited by 23
  27. 2022Normalizing Flows for Interventional Density Estimationcited by 21
  28. 2014Estimating the effect of joint interventions from observational data in sparse high-dimensional settingscited by 20
  29. 2022Scalable Sensitivity and Uncertainty Analysis for Causal-Effect Estimates of Continuous-Valued Interventionscited by 20
  30. 2020GraphITE: Estimating Individual Effects of Graph-structured Treatmentscited by 18
  31. 2022Robust and Agnostic Learning of Conditional Distributional Treatment Effectscited by 18
  32. 2021Generalizing Off-Policy Learning under Sample Selection Biascited by 16
  33. 2021NCoRE: Neural Counterfactual Representation Learning for Combinations of Treatmentscited by 14
  34. 2020Efficient Policy Learning from Surrogate-Loss Classification Reductionscited by 10
  35. 2022Generalization bounds and algorithms for estimating conditional average treatment effect of dosagecited by 9
  36. 2020Regret Minimization for Causal Inference on Large Treatment Spacecited by 7
  37. 2022Estimating Conditional Average Treatment Effects with Missing Treatment Informationcited by 7
  38. 2017Semiparametric Causal Sufficient Dimension Reduction Of Multidimensional Treatmentscited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 6; 2026: 3.

15 papers.

Sort:

  1. 2024Causal machine learning for predicting treatment outcomescited by 74
  2. 2025Foundation Models for Causal Inference via Prior-Data Fitted Networkscited by 30
  3. 2023Bayesian Neural Controlled Differential Equations for Treatment Effect Estimationcited by 26
  4. 2024Conformal Prediction for Causal Effects of Continuous Treatmentscited by 19
  5. 2023A Neural Framework for Generalized Causal Sensitivity Analysiscited by 17
  6. 2024Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environmentscited by 11
  7. 2025Treatment Effect Estimation for Optimal Decision-Makingcited by 7
  8. 2025Efficient and Sharp Off-Policy Learning under Unobserved Confoundingcited by 5
  9. 2024Sources of Gain: Decomposing Performance in Conditional Average Dose Response Estimationcited by 2
  10. 2026Rank-Learner: Orthogonal Ranking of Treatment Effectscited by 2
  11. 2025MOSIC: Model-Agnostic Optimal Subgroup Identification with Multi-Constraint for Improved Reliabilitycited by 1
  12. 2025Doubly Robust Fusion of Many Treatments for Policy Learningcited by 1
3 more
  1. 2025ExOSITO: Explainable Off-Policy Learning with Side Information for Intensive Care Unit Blood Test Orderscited by 0
  2. 2026Barycentric Fused Gromov-Wasserstein Balancing for Causal Inference under Multiple Treatmentscited by 0
  3. 2026Weighted Spline-Expanded Networks with Distributional Balancing for Continuous Treatment Effectscited by 0