# A Map of Bandits for E-commerce

arXiv: https://arxiv.org/abs/2107.00680 · HTML: https://mlsanity.com/paper/2107.00680/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 52 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 16 of 16)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling](https://mlsanity.com/paper/1802.09127/) (2018)
- [Bandits with Knapsacks](https://mlsanity.com/paper/1305.2545/) (2013)
- [Online Learning under Delayed Feedback](https://mlsanity.com/paper/1306.0686/) (2013)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Best-Arm Identification in Linear Bandits](https://mlsanity.com/paper/1409.6110/) (2014)
- [Linear Bandits with Stochastic Delayed Feedback](https://mlsanity.com/paper/1807.02089/) (2018)
- [A Survey on Practical Applications of Multi-Armed and Contextual Bandits](https://mlsanity.com/paper/1904.10040/) (2019)
- [Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting](https://mlsanity.com/paper/1902.01520/) (2019)
- [Efficient Learning in Large-Scale Combinatorial Semi-Bandits](https://mlsanity.com/paper/1406.7443/) (2014)
- [An Efficient Bandit Algorithm for Realtime Multivariate Optimization](https://mlsanity.com/paper/1810.09558/) (2018)
- [Best Arm Identification in Generalized Linear Bandits](https://mlsanity.com/paper/1905.08224/) (2019)
- [Deep Neural Linear Bandits: Overcoming Catastrophic Forgetting through Likelihood Matching](https://mlsanity.com/paper/1901.08612/) (2019)
- [Learning to Rank in the Position Based Model with Bandit Feedback](https://mlsanity.com/paper/2004.13106/) (2020)
- [Genie: An Open Box Counterfactual Policy Estimator for Optimizing Sponsored Search Marketplace](https://mlsanity.com/paper/1808.07251/) (2018)
- [A Bandit Approach to Posterior Dialog Orchestration Under a Budget](https://mlsanity.com/paper/1906.09384/) (2019)

## Built on by (most cited first, 9 of 9)

- [Reward Reports for Reinforcement Learning](https://mlsanity.com/paper/2204.10817/) (2022)
- [Practical Bandits: An Industry Perspective](https://mlsanity.com/paper/2302.01223/) (2023)
- [DISCO: An End-to-End Bandit Framework for Personalised Discount Allocation](https://mlsanity.com/paper/2406.06433/) (2024)
- [Optimal Arm Elimination Algorithms for Combinatorial Bandits](https://mlsanity.com/paper/2510.23992/) (2025)
- [Extending Open Bandit Pipeline to Simulate Industry Challenges](https://mlsanity.com/paper/2209.04147/) (2022)
- [The Digital Transformation in Health: How AI Can Improve the Performance of Health Systems](https://mlsanity.com/paper/2409.16098/) (2024)
- [Adversarial Combinatorial Semi-bandits with Graph Feedback](https://mlsanity.com/paper/2502.18826/) (2025)
- [A Nonparametric Contextual Bandit with Arm-level Eligibility Control for Customer Service Routing](https://mlsanity.com/paper/2209.05278/) (2022)
- [Design and Evaluation of Whole-Page Experience Optimization for E-commerce Search](https://mlsanity.com/paper/2602.02514/) (2026)

## Abstract (arXiv)

The rich body of Bandit literature not only offers a diverse toolbox of algorithms, but also makes it hard for a practitioner to find the right solution to solve the problem at hand. Typical textbooks on Bandits focus on designing and analyzing algorithms, and surveys on applications often present a list of individual applications. While these are valuable resources, there exists a gap in mapping applications to appropriate Bandit algorithms. In this paper, we aim to reduce this gap with a structured map of Bandits to help practitioners navigate to find relevant and practical Bandit algorithms. Instead of providing a comprehensive overview, we focus on a small number of key decision points related to reward, action, and features, which often affect how Bandit algorithms are chosen in practice.