# Breaking Determinism: Stochastic Modeling for Reliable Off-Policy Evaluation in Ad Auctions

arXiv: https://arxiv.org/abs/2512.03354 · HTML: https://mlsanity.com/paper/2512.03354/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Builds on (most cited first, 17 of 17)

- [Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1911.06854/) (2019)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Offline A/B testing for Recommender Systems](https://mlsanity.com/paper/1801.07030/) (2018)
- [A Review of Off-Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2212.06355/) (2022)
- [Evaluating the Robustness of Off-Policy Evaluation](https://mlsanity.com/paper/2108.13703/) (2021)
- [AuctionNet: A Novel Benchmark for Decision-Making in Large-Scale Games](https://mlsanity.com/paper/2412.10798/) (2024)
- [Deep Landscape Forecasting for Real-time Bidding Advertising](https://mlsanity.com/paper/1905.03028/) (2019)
- [Ad-load Balancing via Off-policy Learning in a Content Marketplace](https://mlsanity.com/paper/2309.11518/) (2023)
- [Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies](https://mlsanity.com/paper/2006.03900/) (2020)
- [Debiased Off-Policy Evaluation for Recommendation Systems](https://mlsanity.com/paper/2002.08536/) (2020)
- [Long-term Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2404.15691/) (2024)
- [Scalable Bid Landscape Forecasting in Real-time Bidding](https://mlsanity.com/paper/2001.06587/) (2020)
- [Local Metric Learning for Off-Policy Evaluation in Contextual Bandits with Continuous Actions](https://mlsanity.com/paper/2210.13373/) (2022)
- [Real-time Bidding Strategy in Display Advertising: An Empirical Analysis](https://mlsanity.com/paper/2212.02222/) (2022)
- [Arbitrary Distribution Modeling with Censorship in Real-Time Bidding Advertising](https://mlsanity.com/paper/2110.13587/) (2021)
- [Counterfactual Learning with General Data-generating Policies](https://mlsanity.com/paper/2212.01925/) (2022)
- [Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies](https://mlsanity.com/paper/2405.18792/) (2024)

## Built on by (most cited first, 2 of 2)

- [When Can You Trust Offline Evaluation of Equal-Cost Top-k Allocation? A Controlled, Reproducible Benchmark and Practitioner's Guide](https://mlsanity.com/paper/2608.12489/) (2026)
- [BAFF: Bid-Aware Filter Family for Mitigating Training Data Interference in RTB A/B Tests](https://mlsanity.com/paper/2609.08725/) (2026)

## Abstract (arXiv)

Online A/B testing, the gold standard for evaluating new advertising policies, consumes substantial engineering resources and risks significant revenue loss from deploying underperforming variations. This motivates the use of Off-Policy Evaluation (OPE) for rapid, offline assessment. However, applying OPE to ad auctions is fundamentally more challenging than in domains like recommender systems, where stochastic policies are common. In online ad auctions, it is common for the highest-bidding ad to win the impression, resulting in a deterministic, winner-takes-all setting. This results in zero probability of exposure for non-winning ads, rendering standard OPE estimators inapplicable. We introduce the first principled framework for OPE in deterministic auctions by repurposing the bid landscape model to approximate the propensity score. This model allows us to derive robust approximate propensity scores, enabling the use of stable estimators like Self-Normalized Inverse Propensity Scoring (SNIPS) for counterfactual evaluation. We validate our approach on the AuctionNet simulation benchmark and against 2-weeks online A/B test from a large-scale industrial platform. Our method shows remarkable alignment with online results, achieving a 92% Mean Directional Accuracy (MDA) in CTR prediction, significantly outperforming the parametric baseline. MDA is the most critical metric for guiding deployment decisions, as it reflects the ability to correctly predict whether a new model will improve or harm performance. This work contributes the first practical and validated framework for reliable OPE in deterministic auction environments, offering an efficient alternative to costly and risky online experiments.