# Extending Open Bandit Pipeline to Simulate Industry Challenges

arXiv: https://arxiv.org/abs/2209.04147 · HTML: https://mlsanity.com/paper/2209.04147/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 9 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Online Learning under Delayed Feedback](https://mlsanity.com/paper/1306.0686/) (2013)
- [RecSim: A Configurable Simulation Platform for Recommender Systems](https://mlsanity.com/paper/1909.04847/) (2019)
- [Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learning](https://mlsanity.com/paper/1805.10000/) (2018)
- [Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation](https://mlsanity.com/paper/2008.07146/) (2020)
- [RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising](https://mlsanity.com/paper/1808.00720/) (2018)
- [Efficient Contextual Bandits in Non-stationary Worlds](https://mlsanity.com/paper/1708.01799/) (2017)
- [Nonstochastic Multiarmed Bandits with Unrestricted Delays](https://mlsanity.com/paper/1906.00670/) (2019)
- [Stochastic Multi-Armed Bandits with Unrestricted Delay Distributions](https://mlsanity.com/paper/2106.02436/) (2021)
- [Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction](https://mlsanity.com/paper/1907.06558/) (2019)
- [Blocking Bandits](https://mlsanity.com/paper/1907.11975/) (2019)
- [Combining policy gradient and Q-learning](https://mlsanity.com/paper/1611.01626/) (2016)
- [Unifying Online and Counterfactual Learning to Rank](https://mlsanity.com/paper/2012.04426/) (2020)
- [Bayesian Meta-Prior Learning Using Empirical Bayes](https://mlsanity.com/paper/2002.01129/) (2020)
- [To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions](https://mlsanity.com/paper/1907.06412/) (2019)
- [Contextual Blocking Bandits](https://mlsanity.com/paper/2003.03426/) (2020)
- [A Map of Bandits for E-commerce](https://mlsanity.com/paper/2107.00680/) (2021)
- [Ballooning Multi-Armed Bandits](https://mlsanity.com/paper/2001.10055/) (2020)
- [Contextual Bandit Applications in Customer Support Bot](https://mlsanity.com/paper/2112.03210/) (2021)
- [Non-Stationary Dueling Bandits](https://mlsanity.com/paper/2202.00935/) (2022)
- [Simulations in Recommender Systems: An industry perspective](https://mlsanity.com/paper/2109.06723/) (2021)
- [Machine Learning for Fraud Detection in E-Commerce: A Research Agenda](https://mlsanity.com/paper/2107.01979/) (2021)
- [A Linear Bandit for Seasonal Environments](https://mlsanity.com/paper/2004.13576/) (2020)
- [From Clicks to Conversions: Recommendation for long-term reward](https://mlsanity.com/paper/2009.00497/) (2020)

## Built on by (most cited first, 2 of 2)

- [Practical Bandits: An Industry Perspective](https://mlsanity.com/paper/2302.01223/) (2023)
- [Offline Policy Evaluation as a decision support tool for designing Adaptive Experiments](https://mlsanity.com/paper/2609.30273/) (2026)

## Abstract (arXiv)

Bandit algorithms are often used in the e-commerce industry to train Machine Learning (ML) systems when pre-labeled data is unavailable. However, the industry setting poses various challenges that make implementing bandit algorithms in practice non-trivial. In this paper, we elaborate on the challenges of off-policy optimisation, delayed reward, concept drift, reward design, and business rules constraints that practitioners at Booking.com encounter when applying bandit algorithms. Our main contributions is an extension to the Open Bandit Pipeline (OBP) framework. We provide simulation components for some of the above-mentioned challenges to provide future practitioners, researchers, and educators with a resource to address challenges encountered in the e-commerce industry.