# Budget-Constrained Causal Bandits: Bridging Uplift Modeling and Sequential Decision-Making

arXiv: https://arxiv.org/abs/2604.26169 · HTML: https://mlsanity.com/paper/2604.26169/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 28 Apr 2026.

## Builds on (most cited first, 21 of 21)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [Uplift Modeling for Multiple Treatments with Cost Optimization](https://mlsanity.com/paper/1908.05372/) (2019)
- [Thompson Sampling for Budgeted Multi-armed Bandits](https://mlsanity.com/paper/1505.00146/) (2015)
- [E-Commerce Promotions Personalization via Online Multiple-Choice Knapsack with Uplift Modeling](https://mlsanity.com/paper/2108.13298/) (2021)
- [Direct Heterogeneous Causal Learning for Resource Allocation Problems in Marketing](https://mlsanity.com/paper/2211.15728/) (2022)
- [Free Lunch! Retrospective Uplift Modeling for Dynamic Promotions Recommendation within ROI Constraints](https://mlsanity.com/paper/2008.06293/) (2020)
- [Contextual Multi-Armed Bandits for Causal Marketing](https://mlsanity.com/paper/1810.01859/) (2018)
- [LBCF: A Large-Scale Budget-Constrained Causal Forest Algorithm](https://mlsanity.com/paper/2201.12585/) (2022)
- [End-to-End Cost-Effective Incentive Recommendation under Budget Constraint with Uplift Modeling](https://mlsanity.com/paper/2408.11623/) (2024)
- [Multi-armed Bandits with Cost Subsidy](https://mlsanity.com/paper/2011.01488/) (2020)
- [DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMs](https://mlsanity.com/paper/2601.14711/) (2026)
- [Benchmarking for Deep Uplift Modeling in Online Marketing](https://mlsanity.com/paper/2406.00335/) (2024)
- [DISCO: An End-to-End Bandit Framework for Personalised Discount Allocation](https://mlsanity.com/paper/2406.06433/) (2024)
- [Uplifting Bandits](https://mlsanity.com/paper/2206.04091/) (2022)
- [Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics](https://mlsanity.com/paper/2502.01867/) (2025)
- [Bi-Level Decision-Focused Causal Learning for Large-Scale Marketing Optimization: Bridging Observational and Experimental Data](https://mlsanity.com/paper/2510.19517/) (2025)
- [Improving Thompson Sampling via Information Relaxation for Budgeted Multi-armed Bandits](https://mlsanity.com/paper/2408.15535/) (2024)

## Abstract (arXiv)

Treatment allocation under budget constraints is a central challenge in digital advertising. The standard approach trains an offline uplift model on historical data, then solves a constrained optimization to allocate budget. This fails in cold-start settings where little historical data exists. We propose Budget-Constrained Causal Bandits (BCCB), an online framework that learns which users respond to ads while simultaneously spending the budget. BCCB unifies three components: learning individual-level treatment effects, exploring users whose response is uncertain, and pacing the budget over time. We derive the per-arrival decision rule as the KKT condition of a Lagrangian relaxation of the budgeted causal-allocation objective, providing a principled foundation for the algorithm. We evaluate on the Criteo Uplift dataset using 20 random seeds with paired statistical tests. Our central finding is a data-efficiency crossover at n = 7,500 historical observations (paired one-sided t-test, p = 0.043): below this threshold, offline pipelines either fail or produce unreliable allocations, while BCCB operates from the first user. BCCB exhibits 2-4x lower run-to-run variance than offline methods and outperforms all four online baselines (Thompson Sampling, budgeted Thompson Sampling, HTE Greedy, and Uplifting Bandits) at every budget level tested (p < 0.001). These results give practitioners a concrete decision rule for choosing between offline and online paradigms.