# Risk averse non-stationary multi-armed bandits

arXiv: https://arxiv.org/abs/2109.13977 · HTML: https://mlsanity.com/paper/2109.13977/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 28 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [A Survey on Practical Applications of Multi-Armed and Contextual Bandits](https://mlsanity.com/paper/1904.10040/) (2019)
- [Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Bandits](https://mlsanity.com/paper/1401.1123/) (2014)
- [Optimal Thompson Sampling strategies for support-aware CVaR bandits](https://mlsanity.com/paper/2012.05754/) (2020)
- [Concentration bounds for empirical conditional value-at-risk: The unbounded case](https://mlsanity.com/paper/1808.01739/) (2018)
- [Risk-Constrained Thompson Sampling for CVaR Bandits](https://mlsanity.com/paper/2011.08046/) (2020)
- [Statistically Robust, Risk-Averse Best Arm Identification in Multi-Armed Bandits](https://mlsanity.com/paper/2008.13629/) (2020)
- [Bias-Corrected Peaks-Over-Threshold Estimation of the CVaR](https://mlsanity.com/paper/2103.05059/) (2021)
- [Concentration bounds for CVaR estimation: The cases of light-tailed and heavy-tailed distributions](https://mlsanity.com/paper/1901.00997/) (2019)
- [Risk-Averse Action Selection Using Extreme Value Theory Estimates of the CVaR](https://mlsanity.com/paper/1912.01718/) (2019)

## Built on by (most cited first, 1 of 1)

- [Choquet regularization for reinforcement learning](https://mlsanity.com/paper/2208.08497/) (2022)

## Abstract (arXiv)

This paper tackles the risk averse multi-armed bandits problem when incurred losses are non-stationary. The conditional value-at-risk (CVaR) is used as the objective function. Two estimation methods are proposed for this objective function in the presence of non-stationary losses, one relying on a weighted empirical distribution of losses and another on the dual representation of the CVaR. Such estimates can then be embedded into classic arm selection methods such as epsilon-greedy policies. Simulation experiments assess the performance of the arm selection algorithms based on the two novel estimation approaches, and such policies are shown to outperform naive benchmarks not taking non-stationarity into account.