# Adaptive Exploration-Exploitation Tradeoff for Opportunistic Bandits

arXiv: https://arxiv.org/abs/1709.04004 · HTML: https://mlsanity.com/paper/1709.04004/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2017.

## Built on by (most cited first, 8 of 8)

- [Online V2X Scheduling for Raw-Level Cooperative Perception](https://mlsanity.com/paper/2202.06085/) (2022)
- [Exploration, Exploitation, and Engagement in Multi-Armed Bandits with Abandonment](https://mlsanity.com/paper/2205.13566/) (2022)
- [Knowledge-Guided Exploration in Deep Reinforcement Learning](https://mlsanity.com/paper/2210.15670/) (2022)
- [PAUSE: Low-Latency and Privacy-Aware Active User Selection for Federated Learning](https://mlsanity.com/paper/2503.13173/) (2025)
- [Adaptive Bandit Algorithms for Contextual Matching Markets](https://mlsanity.com/paper/2605.28290/) (2026)
- [Learning-based decentralized offloading decision making in an adversarial environment](https://mlsanity.com/paper/2104.12827/) (2021)
- [Opportunistic Episodic Reinforcement Learning](https://mlsanity.com/paper/2210.13504/) (2022)
- [Partially-Observable Sequential Change-Point Detection for Autocorrelated Data via Upper Confidence Region](https://mlsanity.com/paper/2404.00220/) (2024)

## Abstract (arXiv)

In this paper, we propose and study opportunistic bandits - a new variant of bandits where the regret of pulling a suboptimal arm varies under different environmental conditions, such as network load or produce price. When the load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g., trying a suboptimal network configuration). Therefore, intuitively, we could explore more when the load/price is low and exploit more when the load/price is high. Inspired by this intuition, we propose an Adaptive Upper-Confidence-Bound (AdaUCB) algorithm to adaptively balance the exploration-exploitation tradeoff for opportunistic bandits. We prove that AdaUCB achieves O(log T) regret with a smaller coefficient than the traditional UCB algorithm. Furthermore, AdaUCB achieves O(1) regret with respect to T if the exploration cost is zero when the load level is below a certain threshold. Last, based on both synthetic data and real-world traces, experimental results show that AdaUCB significantly outperforms other bandit algorithms, such as UCB and TS (Thompson Sampling), under large load/price fluctuations.