# Counterfactual Learning with General Data-generating Policies

arXiv: https://arxiv.org/abs/2212.01925 · HTML: https://mlsanity.com/paper/2212.01925/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 4 Dec 2022.

## Builds on (most cited first, 8 of 8)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Doubly Robust Policy Evaluation and Optimization](https://mlsanity.com/paper/1503.02834/) (2015)
- [More Robust Doubly Robust Off-policy Evaluation](https://mlsanity.com/paper/1802.03493/) (2018)
- [Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation](https://mlsanity.com/paper/2002.09516/) (2020)
- [Doubly robust off-policy evaluation with shrinkage](https://mlsanity.com/paper/1907.09623/) (2019)
- [Off-policy Bandits with Deficient Support](https://mlsanity.com/paper/2006.09438/) (2020)
- [Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting](https://mlsanity.com/paper/2006.10460/) (2020)
- [Algorithm as Experiment: Machine Learning, Market Design, and Policy Eligibility Rules](https://mlsanity.com/paper/2104.12909/) (2021)

## Built on by (most cited first, 3 of 3)

- [Breaking Determinism: Stochastic Modeling for Reliable Off-Policy Evaluation in Ad Auctions](https://mlsanity.com/paper/2512.03354/) (2025)
- [CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning](https://mlsanity.com/paper/2512.09368/) (2025)
- [When Does Action Credit Need Updating?](https://mlsanity.com/paper/2609.29007/) (2026)

## Abstract (arXiv)

Off-policy evaluation (OPE) attempts to predict the performance of counterfactual policies using log data from a different policy. We extend its applicability by developing an OPE method for a class of both full support and deficient support logging policies in contextual-bandit settings. This class includes deterministic bandit (such as Upper Confidence Bound) as well as deterministic decision-making based on supervised and unsupervised learning. We prove that our method's prediction converges in probability to the true performance of a counterfactual policy as the sample size increases. We validate our method with experiments on partly and entirely deterministic logging policies. Finally, we apply it to evaluate coupon targeting policies by a major online platform and show how to improve the existing policy.