# Online Learning for Cooperative Multi-Player Multi-Armed Bandits

arXiv: https://arxiv.org/abs/2109.03818 · HTML: https://mlsanity.com/paper/2109.03818/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 8 of 8)

- [Distributed Clustering of Linear Bandits in Peer to Peer Networks](https://mlsanity.com/paper/1604.07706/) (2016)
- [Collaborative Learning with Limited Interaction: Tight Bounds for Distributed Exploration in Multi-Armed Bandits](https://mlsanity.com/paper/1904.03293/) (2019)
- [Decentralized Multi-player Multi-armed Bandits with No Collision Information](https://mlsanity.com/paper/2003.00162/) (2020)
- [Concurrent bandits and cognitive radio networks](https://mlsanity.com/paper/1404.5421/) (2014)
- [Collaborative Top Distribution Identifications with Limited Interaction](https://mlsanity.com/paper/2004.09454/) (2020)
- [On Regret-Optimal Learning in Decentralized Multi-player Multi-armed Bandits](https://mlsanity.com/paper/1505.00553/) (2015)
- [Decentralized Exploration in Multi-Armed Bandits -- Extended version](https://mlsanity.com/paper/1811.07763/) (2018)
- [Channel Selection for Network-assisted D2D Communication via No-Regret Bandit Learning with Calibrated Forecasting](https://mlsanity.com/paper/1404.7061/) (2014)

## Built on by (most cited first, 12 of 12)

- [On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2110.05707/) (2021)
- [Decentralized Cooperative Reinforcement Learning with Hierarchical Information Structure](https://mlsanity.com/paper/2111.00781/) (2021)
- [Optimal Cooperative Multiplayer Learning Bandits with Noisy Rewards and No Communication](https://mlsanity.com/paper/2311.06210/) (2023)
- [Multiplayer Information Asymmetric Bandits in Metric Spaces](https://mlsanity.com/paper/2503.08004/) (2025)
- [Multiplayer Information Asymmetric Contextual Bandits](https://mlsanity.com/paper/2503.08961/) (2025)
- [Cost of Structural Learning Under Censored Feedback: A Threshold-Bandit Approach](https://mlsanity.com/paper/2605.27076/) (2026)
- [Coordinating the Unknown Lipschitz Constant in Multiplayer Bandits](https://mlsanity.com/paper/2608.10526/) (2026)
- [Robust Multi-Agent Bandits with Heavy-Tailed Rewards and Information Asymmetry](https://mlsanity.com/paper/2608.10529/) (2026)
- [DCM Bandits: Multiplayer Information Asymmetric Cascading Bandits for Multiple Clicks](https://mlsanity.com/paper/2608.11873/) (2026)
- [Decentralized Multi-Player Q-Learning in Episodic Markov Decision Processes with Information Asymmetry](https://mlsanity.com/paper/2608.12753/) (2026)
- [The Price of Decentralization in Top-K Arm Identification](https://mlsanity.com/paper/2608.22120/) (2026)
- [Improved Multiplayer Bandit Algorithm for Bernoulli Rewards](https://mlsanity.com/paper/2609.26213/) (2026)

## Abstract (arXiv)

We introduce a framework for decentralized online learning for multi-armed bandits (MAB) with multiple cooperative players. The reward obtained by the players in each round depends on the actions taken by all the players. It's a team setting, and the objective is common. Information asymmetry is what makes the problem interesting and challenging. We consider three types of information asymmetry: action information asymmetry when the actions of the players can't be observed but the rewards received are common; reward information asymmetry when the actions of the other players are observable but rewards received are IID from the same distribution; and when we have both action and reward information asymmetry. For the first setting, we propose a UCB-inspired algorithm that achieves O(log T) regret whether the rewards are IID or Markovian. For the second section, we offer an environment such that the algorithm given for the first setting gives linear regret. For the third setting, we show that a variation of the `explore then commit' algorithm achieves almost log regret.