mlsanityData to 14 Oct 2026

Taming Non-stationary Bandits: A Bayesian Approach

arXiv 1707.09727 ↗First posted 31 Jul 2017stat.ML cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

We consider the multi armed bandit problem in non-stationary environments. Based on the Bayesian method, we propose a variant of Thompson Sampling which can be used in both rested and restless bandit scenarios. Applying discounting to the parameters of prior distribution, we describe a way to systematically reduce the effect of past observations. Further, we derive the exact expression for the probability of picking sub-optimal arms. By increasing the exploitative value of Bayes' samples, we also provide an optimistic version of the algorithm. Extensive empirical analysis is conducted under various scenarios to validate the utility of proposed algorithms. A comparison study with various state-of-the-arm algorithms is also included.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 8; 2024: 5; 2025: 4; 2026: 5.

32 papers.

Sort:

By area:

  1. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  2. 2022Non-Stationary Bandit Learning via Predictive Samplingcited by 21
  3. 2024Sliding-Window Thompson Sampling for Non-Stationary Settingscited by 17
  4. 2025BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuningcited by 14
  5. 2021An empirical evaluation of active inference in multi-armed banditscited by 12
  6. 2023Module-wise Adaptive Distillation for Multimodality Foundation Modelscited by 12
  7. 2023An Information-Theoretic Analysis of Nonstationary Bandit Learningcited by 11
  8. 2022Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learningcited by 9
  9. 2023A Definition of Non-Stationary Banditscited by 9
  10. 2021On Limited-Memory Subsampling Strategies for Banditscited by 8
  11. 2021Nonstationary Stochastic Multiarmed Bandits: UCB Policies and Minimax Regretcited by 6
  12. 2023Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMscited by 5
20 more
  1. 2023Discounted Thompson Sampling for Non-Stationary Bandit Problemscited by 4
  2. 2024Non-Stationary Latent Auto-Regressive Banditscited by 3
  3. 2022Fast Change Identification in Multi-Play Bandits and its Applications in Wireless Networkscited by 2
  4. 2022Safety Aware Changepoint Detection for Piecewise i.i.d. Banditscited by 2
  5. 2021Existence conditions for hidden feedback loops in online recommender systemscited by 1
  6. 2022SLOPT: Bandit Optimization Framework for Mutation-Based Fuzzingcited by 1
  7. 2023Non-Stationary Contextual Bandit Learning via Neural Predictive Ensemble Samplingcited by 1
  8. 2023Forced Exploration in Bandit Problemscited by 1
  9. 2022Dynamic Memory for Interpretable Sequential Optimisationcited by 0
  10. 2024Bridging Rested and Restless Bandits with Graph-Triggering: Rising and Rottingcited by 0
  11. 2024Improving Portfolio Optimization Results with Bandit Networkscited by 0
  12. 2024Diminishing Exploration: A Minimalist Approach to Piecewise Stationary Multi-Armed Banditscited by 0
  13. 2025Natural Policy Gradient for Average Reward Non-Stationary RLcited by 0
  14. 2025Thompson Sampling-Based Learning and Control for Unknown Dynamic Systemscited by 0
  15. 2025TS-Insight: Visualizing Thompson Sampling for Verification and XAIcited by 0
  16. 2026A Modularized Framework for Piecewise-Stationary Restless Banditscited by 0
  17. 2026Flow-Corrected Thompson Sampling for Non-Stationary Contextual Banditscited by 0
  18. 2026Odds-Ratio Thompson Sampling: A Specification and Design Guide for Contrast-Based Multi-Armed Banditscited by 0
  19. 2026When May a Bandit Leave Its Anchor? E-Process-Authorized Thompson Sampling under Non-stationaritycited by 0
  20. 2026Choosing LLMs for Strategic Agent Interactionscited by 0