mlsanityData to 14 Oct 2026

On Abruptly-Changing and Slowly-Varying Multiarmed Bandit Problems

arXiv 1802.08380 ↗First posted 23 Feb 2018stat.ML cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

We study the non-stationary stochastic multiarmed bandit (MAB) problem and propose two generic algorithms, namely, the limited memory deterministic sequencing of exploration and exploitation (LM-DSEE) and the Sliding-Window Upper Confidence Bound# (SW-UCB#). We rigorously analyze these algorithms in abruptly-changing and slowly-varying environments and characterize their performance. We show that the expected cumulative regret for these algorithms under either of the environments is upper bounded by sublinear functions of time, i.e., the time average of the regret asymptotically converges to zero. We complement our analytic results with numerical illustrations.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 3; 2024: 1; 2025: 2; 2026: 1.

15 papers.

Sort:

By area:

  1. 2021Tracking Most Significant Arm Switches in Banditscited by 27
  2. 2022Non-Stationary Bandit Learning via Predictive Samplingcited by 21
  3. 2023Tracking Most Significant Shifts in Nonparametric Contextual Banditscited by 10
  4. 2023A Definition of Non-Stationary Banditscited by 9
  5. 2021Finite-time Analysis of Globally Nonstationary Multi-Armed Banditscited by 7
  6. 2021On Slowly-varying Non-stationary Banditscited by 7
  7. 2021Nonstationary Stochastic Multiarmed Bandits: UCB Policies and Minimax Regretcited by 6
  8. 2022Decentralized Competing Bandits in Non-Stationary Matching Marketscited by 5
  9. 2024Adaptive Smooth Non-Stationary Banditscited by 4
  10. 2023Distributed Consensus Algorithm for Decision-Making in Multi-agent Multi-armed Banditcited by 2
  11. 2022Curriculum optimization for low-resource speech recognitioncited by 1
  12. 2022Deterministic Sequencing of Exploration and Exploitation for Reinforcement Learningcited by 1
3 more
  1. 2025Solve Smart, Not Often: Policy Learning for Costly MILP Re-solvingcited by 1
  2. 2025Fixed-Confidence Best Arm Identification with Decreasing Variancecited by 0
  3. 2026Nonparametric Learning and Earning with One-Point Feedback under Nonstationaritycited by 0