# Optimal Policy Adaptation under Covariate Shift

arXiv: https://arxiv.org/abs/2501.08067 · HTML: https://mlsanity.com/paper/2501.08067/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 53 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Builds on (most cited first, 20 of 20)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Causal inference methods for combining randomized trials and observational studies: a review](https://mlsanity.com/paper/2011.08047/) (2020)
- [On the role of surrogates in the efficient estimation of treatment effects with limited outcome data](https://mlsanity.com/paper/2003.12408/) (2020)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [On the Opportunity of Causal Learning in Recommendation Systems: Foundation, Estimation, Prediction and Challenges](https://mlsanity.com/paper/2201.06716/) (2022)
- [Long-term Causal Inference Under Persistent Confounding via Data Combination](https://mlsanity.com/paper/2202.07234/) (2022)
- [Off-Policy Evaluation and Learning for External Validity under a Covariate Shift](https://mlsanity.com/paper/2002.11642/) (2020)
- [Extending inferences from a randomized trial to a new target population](https://mlsanity.com/paper/1805.00550/) (2018)
- [TDR-CL: Targeted Doubly Robust Collaborative Learning for Debiased Recommendations](https://mlsanity.com/paper/2203.10258/) (2022)
- [Balancing Unobserved Confounding with a Few Unbiased Ratings in Debiased Recommendations](https://mlsanity.com/paper/2304.09085/) (2023)
- [Causal Effect Identification from Multiple Incomplete Data Sources: A General Search-based Approach](https://mlsanity.com/paper/1902.01073/) (2019)
- [Semi-supervised learning for medical image classification using imbalanced training data](https://mlsanity.com/paper/2108.08956/) (2021)
- [Improving efficiency of inference in clinical trials with external control data](https://mlsanity.com/paper/2011.07234/) (2020)
- [Identification and estimation of treatment effects on long-term outcomes in clinical trials with external observational data](https://mlsanity.com/paper/2208.10163/) (2022)
- [Propensity score regression for causal inference with treatment heterogeneity](https://mlsanity.com/paper/2109.07722/) (2021)
- [Policy Learning for Balancing Short-Term and Long-Term Rewards](https://mlsanity.com/paper/2405.03329/) (2024)
- [Quantifying Individual Risk for Binary Outcomes](https://mlsanity.com/paper/2402.10537/) (2024)
- [Generalizing trial evidence to target populations in non-nested designs: Applications to AIDS clinical trials](https://mlsanity.com/paper/2103.04907/) (2021)
- [On the Comparative Analysis of Average Treatment Effects Estimation via Data Combination](https://mlsanity.com/paper/2311.00528/) (2023)

## Abstract (arXiv)

Transfer learning of prediction models has been extensively studied, while the corresponding policy learning approaches are rarely discussed. In this paper, we propose principled approaches for learning the optimal policy in the target domain by leveraging two datasets: one with full information from the source domain and the other from the target domain with only covariates. First, under the setting of covariate shift, we formulate the problem from a perspective of causality and present the identifiability assumptions for the reward induced by a given policy. Then, we derive the efficient influence function and the semiparametric efficiency bound for the reward. Based on this, we construct a doubly robust and semiparametric efficient estimator for the reward and then learn the optimal policy by optimizing the estimated reward. Moreover, we theoretically analyze the bias and the generalization error bound for the learned policy. Extensive experiments demonstrate that the approach not only estimates the reward more accurately but also yields a policy that closely approximates the theoretically optimal policy.