# Competing Bandits: The Perils of Exploration Under Competition

arXiv: https://arxiv.org/abs/2007.10144 · HTML: https://mlsanity.com/paper/2007.10144/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 12 of 12)

- [Beyond log2(T) Regret for Decentralized Bandits in Matching Markets](https://mlsanity.com/paper/2103.07501/) (2021)
- [Competition, Alignment, and Equilibria in Digital Marketplaces](https://mlsanity.com/paper/2208.14423/) (2022)
- [Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition](https://mlsanity.com/paper/2306.14670/) (2023)
- [Exploration and Incentives in Reinforcement Learning](https://mlsanity.com/paper/2103.00360/) (2021)
- [Emergent specialization from participation dynamics and multi-learner retraining](https://mlsanity.com/paper/2206.02667/) (2022)
- [Impact of Decentralized Learning on Player Utilities in Stackelberg Games](https://mlsanity.com/paper/2403.00188/) (2024)
- [Safety vs. Performance: How Multi-Objective Learning Reduces Barriers to Market Entry](https://mlsanity.com/paper/2409.03734/) (2024)
- [Choosing the Better Bandit Algorithm under Data Sharing: When Do A/B Experiments Work?](https://mlsanity.com/paper/2507.11891/) (2025)
- [Strategic Usage in a Multi-Learner Setting](https://mlsanity.com/paper/2401.16422/) (2024)
- [Learner-Private Convex Optimization](https://mlsanity.com/paper/2102.11976/) (2021)
- [Interactive Learning with Pricing for Optimal and Stable Allocations in Markets](https://mlsanity.com/paper/2212.06891/) (2022)
- [Markets for Models](https://mlsanity.com/paper/2503.02946/) (2025)

## Abstract (arXiv)

Most online platforms strive to learn from interactions with users, and many engage in exploration: making potentially suboptimal choices for the sake of acquiring new information. We study the interplay between exploration and competition: how such platforms balance the exploration for learning and the competition for users. Here users play three distinct roles: they are customers that generate revenue, they are sources of data for learning, and they are self-interested agents which choose among the competing platforms. We consider a stylized duopoly model in which two firms face the same multi-armed bandit problem. Users arrive one by one and choose between the two firms, so that each firm makes progress on its bandit problem only if it is chosen. Through a mix of theoretical results and numerical simulations, we study whether and to what extent competition incentivizes the adoption of better bandit algorithms, and whether it leads to welfare increases for users. We find that stark competition induces firms to commit to a "greedy" bandit algorithm that leads to low welfare. However, weakening competition by providing firms with some "free" users incentivizes better exploration strategies and increases welfare. We investigate two channels for weakening the competition: relaxing the rationality of users and giving one firm a first-mover advantage. Our findings are closely related to the "competition vs. innovation" relationship, and elucidate the first-mover advantage in the digital economy.