# What a Data Release Must Preserve for Off-Policy Evaluation

arXiv: https://arxiv.org/abs/2610.15283 · HTML: https://mlsanity.com/paper/2610.15283/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 11 Oct 2026.

## Builds on (most cited first, 15 of 15)

- [KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems](https://mlsanity.com/paper/2202.10842/) (2022)
- [A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2111.06784/) (2021)
- [Conditional Importance Sampling for Off-Policy Learning](https://mlsanity.com/paper/1910.07479/) (2019)
- [Partial Identification with Noisy Covariates: A Robust Optimization Approach](https://mlsanity.com/paper/2202.10665/) (2022)
- [Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory](https://mlsanity.com/paper/2605.10870/) (2026)
- [State-Action Similarity-Based Representations for Off-Policy Evaluation](https://mlsanity.com/paper/2310.18409/) (2023)
- [When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspective](https://mlsanity.com/paper/2311.14110/) (2023)
- [Off-policy Evaluation with Deeply-abstracted States](https://mlsanity.com/paper/2406.19531/) (2024)
- [Logging Policy Design for Off-Policy Evaluation](https://mlsanity.com/paper/2605.15108/) (2026)
- [Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluation](https://mlsanity.com/paper/2410.02172/) (2024)
- [Bridging multiple worlds: multi-marginal optimal transport for causal partial-identification problem](https://mlsanity.com/paper/2406.07868/) (2024)
- [The hidden risks of temporal resampling in clinical reinforcement learning](https://mlsanity.com/paper/2602.06603/) (2026)
- [Target-Oriented Statistical Compression: Sufficiency, Reverse Martingales, and Sequential Monitoring](https://mlsanity.com/paper/2605.26568/) (2026)
- [Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensities](https://mlsanity.com/paper/2605.29500/) (2026)
- [Privacy-Robust Incrementality Measurement for Advertising Systems under Signal Loss](https://mlsanity.com/paper/2606.03878/) (2026)

## Abstract (arXiv)

In off-policy evaluation, merging decision-time histories that matter differently for a specified policy value or comparison can make that evaluation unidentified, even with observed actions and outcomes and known behavior and target policies. We characterize which distinctions must be preserved: identification holds if and only if the action-specific evaluation weights are constant within each released record, and the coarsest exact release groups histories by these weights. For normalized outcomes, the largest sharp width induced by the release equals a behavior-weighted within-record radius of these weights. The same radius becomes a release-design objective when each record may be refined into at most m categories. The optimum has an exact m-center characterization. When behavior does not vary within a released record, we give a finite-data repair algorithm whose maximum evaluation-weight radius is at most twice the finite-data optimum. For a fixed action-outcome distribution within a released record, a distinct sharp-identification problem remains. Under the same condition, the action-specific outcome distributions are identified, and optimizing over their couplings gives the sharp interval. We construct an AIPW envelope containing this sharp interval; it matches the interval for all such outcome distributions exactly when the extreme within-record action-coefficient vectors take at most two values in each action coordinate. In 24 KuaiRec tasks, the repair algorithm yields a 93% median reduction in mean pairwise sharp width at m=64 relative to the unrepaired release.