# Off-Policy Evaluation and Learning for External Validity under a Covariate Shift

arXiv: https://arxiv.org/abs/2002.11642 · HTML: https://mlsanity.com/paper/2002.11642/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 26 of 26)

- [Evaluating the Robustness of Off-Policy Evaluation](https://mlsanity.com/paper/2108.13703/) (2021)
- [Generalizing Off-Policy Learning under Sample Selection Bias](https://mlsanity.com/paper/2112.01387/) (2021)
- [Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings](https://mlsanity.com/paper/2105.06029/) (2021)
- [Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choices](https://mlsanity.com/paper/2403.03589/) (2024)
- [Doubly Robust Alignment for Large Language Models](https://mlsanity.com/paper/2506.01183/) (2025)
- [Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage](https://mlsanity.com/paper/2302.02392/) (2023)
- [Combining Online Learning and Offline Learning for Contextual Bandits with Deficient Support](https://mlsanity.com/paper/2107.11533/) (2021)
- [Learning Complementary Policies for Human-AI Teams](https://mlsanity.com/paper/2302.02944/) (2023)
- [Nearest Neighbor Matching as Least Squares Density Ratio Estimation and Riesz Regression](https://mlsanity.com/paper/2510.24433/) (2025)
- [Federated Learning under Covariate Shifts with Generalization Guarantees](https://mlsanity.com/paper/2306.05325/) (2023)
- [Double Debiased Covariate Shift Adaptation Robust to Density-Ratio Estimation](https://mlsanity.com/paper/2310.16638/) (2023)
- [Direct Debiased Machine Learning via Bregman Divergence Minimization](https://mlsanity.com/paper/2510.23534/) (2025)
- [Distributionally Robust Policy Evaluation under General Covariate Shift in Contextual Bandits](https://mlsanity.com/paper/2401.11353/) (2024)
- [Learning Under Adversarial and Interventional Shifts](https://mlsanity.com/paper/2103.15933/) (2021)
- [MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks](https://mlsanity.com/paper/2305.13271/) (2023)
- [Counterfactual Learning with Multioutput Deep Kernels](https://mlsanity.com/paper/2211.11119/) (2022)
- [Two-sample comparison through additive tree models for density ratios](https://mlsanity.com/paper/2508.03059/) (2025)
- [Semi-Supervised Treatment Effect Estimation with Unlabeled Covariates for Prediction-Powered Causal Inference](https://mlsanity.com/paper/2511.08303/) (2025)
- [Rejoinder: Learning Optimal Distributionally Robust Individualized Treatment Rules](https://mlsanity.com/paper/2110.08936/) (2021)
- [Unified Perspective on Probability Divergence via Maximum Likelihood Density Ratio Estimation: Bridging KL-Divergence and Integral Probability Metrics](https://mlsanity.com/paper/2201.13127/) (2022)
- [Towards Robust Off-Policy Evaluation via Human Inputs](https://mlsanity.com/paper/2209.08682/) (2022)
- [Bayesian Counterfactual Mean Embeddings and Off-Policy Evaluation](https://mlsanity.com/paper/2211.01518/) (2022)
- [Sample Complexity of Preference-Based Nonparametric Off-Policy Evaluation with Deep Networks](https://mlsanity.com/paper/2310.10556/) (2023)
- [Optimal Policy Adaptation under Covariate Shift](https://mlsanity.com/paper/2501.08067/) (2025)
- [Off-Policy Evaluation with Strategic Agents via Local Disclosure](https://mlsanity.com/paper/2606.07308/) (2026)
- [Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits](https://mlsanity.com/paper/2607.22012/) (2026)

## Abstract (arXiv)

We consider evaluating and training a new policy for the evaluation data by using the historical data obtained from a different policy. The goal of off-policy evaluation (OPE) is to estimate the expected reward of a new policy over the evaluation data, and that of off-policy learning (OPL) is to find a new policy that maximizes the expected reward over the evaluation data. Although the standard OPE and OPL assume the same distribution of covariate between the historical and evaluation data, a covariate shift often exists, i.e., the distribution of the covariate of the historical data is different from that of the evaluation data. In this paper, we derive the efficiency bound of OPE under a covariate shift. Then, we propose doubly robust and efficient estimators for OPE and OPL under a covariate shift by using a nonparametric estimator of the density ratio between the historical and evaluation data distributions. We also discuss other possible estimators and compare their theoretical properties. Finally, we confirm the effectiveness of the proposed estimators through experiments.