# Balanced Policy Evaluation and Learning

arXiv: https://arxiv.org/abs/1705.07384 · HTML: https://mlsanity.com/paper/1705.07384/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2017.

## Built on by (most cited first, 30 of 52)

- [Post-Contextual-Bandit Inference](https://mlsanity.com/paper/2106.00418/) (2021)
- [Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequality](https://mlsanity.com/paper/2212.09900/) (2022)
- [Estimating Average Treatment Effects via Orthogonal Regularization](https://mlsanity.com/paper/2101.08490/) (2021)
- [Policy Learning with Adaptively Collected Data](https://mlsanity.com/paper/2105.02344/) (2021)
- [Invariant Policy Learning: A Causal Perspective](https://mlsanity.com/paper/2106.00808/) (2021)
- [Generalizing Off-Policy Learning under Sample Selection Bias](https://mlsanity.com/paper/2112.01387/) (2021)
- [Reliable Off-Policy Learning for Dosage Combinations](https://mlsanity.com/paper/2305.19742/) (2023)
- [Oracle-Efficient Pessimism: Offline Policy Optimization in Contextual Bandits](https://mlsanity.com/paper/2306.07923/) (2023)
- [Treatment Effect Risk: Bounds and Inference](https://mlsanity.com/paper/2201.05893/) (2022)
- [Inference on Strongly Identified Functionals of Weakly Identified Functions](https://mlsanity.com/paper/2208.08291/) (2022)
- [Fair Off-Policy Learning from Observational Data](https://mlsanity.com/paper/2303.08516/) (2023)
- [Learning Optimal Prescriptive Trees from Observational Data](https://mlsanity.com/paper/2108.13628/) (2021)
- [Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment](https://mlsanity.com/paper/2109.11679/) (2021)
- [Uncertainty Quantification for Fairness in Two-Stage Recommender Systems](https://mlsanity.com/paper/2205.15436/) (2022)
- [Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap](https://mlsanity.com/paper/2104.05762/) (2021)
- [Policy Learning with Asymmetric Counterfactual Utilities](https://mlsanity.com/paper/2206.10479/) (2022)
- [Causal Fairness for Outcome Control](https://mlsanity.com/paper/2306.05066/) (2023)
- [Treatment Effect Estimation for Optimal Decision-Making](https://mlsanity.com/paper/2505.13092/) (2025)
- [Learning Complementary Policies for Human-AI Teams](https://mlsanity.com/paper/2302.02944/) (2023)
- [Loss Functions for Discrete Contextual Pricing with Observational Data](https://mlsanity.com/paper/2111.09933/) (2021)
- [Kernel-based off-policy estimation without overlap: Instance optimality beyond semiparametric efficiency](https://mlsanity.com/paper/2301.06240/) (2023)
- [Fair and Robust Estimation of Heterogeneous Treatment Effects for Policy Learning](https://mlsanity.com/paper/2306.03625/) (2023)
- [Individualized Policy Evaluation and Learning under Clustered Network Interference](https://mlsanity.com/paper/2311.02467/) (2023)
- [Efficient and Sharp Off-Policy Learning under Unobserved Confounding](https://mlsanity.com/paper/2502.13022/) (2025)
- [Balanced Off-Policy Evaluation for Personalized Pricing](https://mlsanity.com/paper/2302.12736/) (2023)
- [Bayesian Safe Policy Learning with Chance Constrained Optimization: Application to Military Security Assessment during the Vietnam War](https://mlsanity.com/paper/2307.08840/) (2023)
- [Confounding-Robust Policy Improvement with Human-AI Teams](https://mlsanity.com/paper/2310.08824/) (2023)
- [Robust Sample Weighting to Facilitate Individualized Treatment Rule Learning for a Target Population](https://mlsanity.com/paper/2105.00581/) (2021)
- [Interpretable Off-Policy Learning via Hyperbox Search](https://mlsanity.com/paper/2203.02473/) (2022)
- [Off-Policy Evaluation with Out-of-Sample Guarantees](https://mlsanity.com/paper/2301.08649/) (2023)

## Abstract (arXiv)

We present a new approach to the problems of evaluating and learning personalized decision policies from observational data of past contexts, decisions, and outcomes. Only the outcome of the enacted decision is available and the historical policy is unknown. These problems arise in personalized medicine using electronic health records and in internet advertising. Existing approaches use inverse propensity weighting (or, doubly robust versions) to make historical outcome (or, residual) data look like it were generated by a new policy being evaluated or learned. But this relies on a plug-in approach that rejects data points with a decision that disagrees with the new policy, leading to high variance estimates and ineffective learning. We propose a new, balance-based approach that too makes the data look like the new policy but does so directly by finding weights that optimize for balance between the weighted data and the target policy in the given, finite sample, which is equivalent to minimizing worst-case or posterior conditional mean square error. Our policy learner proceeds as a two-level optimization problem over policies and weights. We demonstrate that this approach markedly outperforms existing ones both in evaluation and learning, which is unsurprising given the wider support of balance-based weights. We establish extensive theoretical consistency guarantees and regret bounds that support this empirical success.