# Maximizing the Success Probability of Policy Allocations in Online Systems

arXiv: https://arxiv.org/abs/2312.16267 · HTML: https://mlsanity.com/paper/2312.16267/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 26 Dec 2023.

## Builds on (most cited first, 13 of 13)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [A Unifying Framework for Online Optimization with Long-Term Constraints](https://mlsanity.com/paper/2209.07454/) (2022)
- [A Unified Framework for Marketing Budget Allocation](https://mlsanity.com/paper/1902.01128/) (2019)
- [E-Commerce Promotions Personalization via Online Multiple-Choice Knapsack with Uplift Modeling](https://mlsanity.com/paper/2108.13298/) (2021)
- [Multiplicative Pacing Equilibria in Auction Markets](https://mlsanity.com/paper/1706.07151/) (2017)
- [A Large Scale Benchmark for Individual Treatment Effect Prediction and Uplift Modeling](https://mlsanity.com/paper/2111.10106/) (2021)
- [A Comparison of Methods for Treatment Assignment with an Application to Playlist Generation](https://mlsanity.com/paper/2004.11532/) (2020)
- [An End-to-End Framework for Marketing Effectiveness Optimization under Budget Constraint](https://mlsanity.com/paper/2302.04477/) (2023)
- [A Real-World Implementation of Unbiased Lift-based Bidding System](https://mlsanity.com/paper/2202.13868/) (2022)

## Built on by (most cited first, 4 of 4)

- [Position Paper: Why the Shooting in the Dark Method Dominates Recommender Systems Practice; A Call to Abandon Anti-Utopian Thinking](https://mlsanity.com/paper/2402.02152/) (2024)
- [Hidden Representation Clustering with Multi-Task Representation Learning towards Robust Online Budget Allocation](https://mlsanity.com/paper/2506.00959/) (2025)
- [Maximizing Reliability with Bayesian Optimization](https://mlsanity.com/paper/2602.02432/) (2026)
- [Learning to Bid in Repeated Second-Price Auctions with Dynamic Values and Aggregated Feedback](https://mlsanity.com/paper/2605.28133/) (2026)

## Abstract (arXiv)

The effectiveness of advertising in e-commerce largely depends on the ability of merchants to bid on and win impressions for their targeted users. The bidding procedure is highly complex due to various factors such as market competition, user behavior, and the diverse objectives of advertisers. In this paper we consider the problem at the level of user timelines instead of individual bid requests, manipulating full policies (i.e. pre-defined bidding strategies) and not bid values. In order to optimally allocate policies to users, typical multiple treatments allocation methods solve knapsack-like problems which aim at maximizing an expected value under constraints. In the industrial contexts such as online advertising, we argue that optimizing for the probability of success is a more suited objective than expected value maximization, and we introduce the SuccessProbaMax algorithm that aims at finding the policy allocation which is the most likely to outperform a fixed reference policy. Finally, we conduct comprehensive experiments both on synthetic and real-world data to evaluate its performance. The results demonstrate that our proposed algorithm outperforms conventional expected-value maximization algorithms in terms of success rate.