# Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Bandits

arXiv: https://arxiv.org/abs/1401.1123 · HTML: https://mlsanity.com/paper/1401.1123/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2014.

## Built on by (most cited first, 26 of 26)

- [Learning to Map for Active Semantic Goal Navigation](https://mlsanity.com/paper/2106.15648/) (2021)
- [Reinforcement Learning with Dynamic Convex Risk Measures](https://mlsanity.com/paper/2112.13414/) (2021)
- [The Fragility of Optimized Bandit Algorithms](https://mlsanity.com/paper/2109.13595/) (2021)
- [Safe Active Learning for Time-Series Modeling with Gaussian Processes](https://mlsanity.com/paper/2402.06276/) (2024)
- [A Unifying Theory of Thompson Sampling for Continuous Risk-Averse Bandits](https://mlsanity.com/paper/2108.11345/) (2021)
- [Spectral risk-based learning using unbounded losses](https://mlsanity.com/paper/2105.04816/) (2021)
- [Model-Free Risk-Sensitive Reinforcement Learning](https://mlsanity.com/paper/2111.02907/) (2021)
- [ESCADA: Efficient Safety and Context Aware Dose Allocation for Precision Medicine](https://mlsanity.com/paper/2111.13415/) (2021)
- [The Survival Bandit Problem](https://mlsanity.com/paper/2206.03019/) (2022)
- [Safe Online Bid Optimization with Return on Investment and Budget Constraints](https://mlsanity.com/paper/2201.07139/) (2022)
- [Continuous Mean-Covariance Bandits](https://mlsanity.com/paper/2102.12090/) (2021)
- [Thompson Sampling for Gaussian Entropic Risk Bandits](https://mlsanity.com/paper/2105.06960/) (2021)
- [Risk-Aware Algorithms for Combinatorial Semi-Bandits](https://mlsanity.com/paper/2112.01141/) (2021)
- [Risk averse non-stationary multi-armed bandits](https://mlsanity.com/paper/2109.13977/) (2021)
- [Risk-averse Contextual Multi-armed Bandit Problem with Linear Payoffs](https://mlsanity.com/paper/2206.12463/) (2022)
- [Risk-aware linear bandits with convex loss](https://mlsanity.com/paper/2209.07154/) (2022)
- [Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processes](https://mlsanity.com/paper/2405.10581/) (2024)
- [Constrained Online Decision-Making: A Unified Framework](https://mlsanity.com/paper/2505.07101/) (2025)
- [Order Optimal Regret Bounds for Sharpe Ratio Optimization under Thompson Sampling](https://mlsanity.com/paper/2508.13749/) (2025)
- [Understanding the stochastic dynamics of sequential decision-making processes: A path-integral analysis of multi-armed bandits](https://mlsanity.com/paper/2208.06245/) (2022)
- [Balancing Risk and Reward: An Automated Phased Release Strategy](https://mlsanity.com/paper/2305.09626/) (2023)
- [A Risk-Averse Framework for Non-Stationary Stochastic Multi-Armed Bandits](https://mlsanity.com/paper/2310.19821/) (2023)
- [Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability](https://mlsanity.com/paper/2512.09850/) (2025)
- [Risk-Averse Wasserstein Distributionally Robust Online Learning](https://mlsanity.com/paper/2602.20403/) (2026)
- [Concave Statistical Utility Maximization Bandits via Influence-Function Gradients](https://mlsanity.com/paper/2604.22140/) (2026)
- [Conformal-Style Quantile Analyses for Stochastic Bandits](https://mlsanity.com/paper/2605.07115/) (2026)

## Abstract (arXiv)

Motivated by applications in energy management, this paper presents the Multi-Armed Risk-Aware Bandit (MARAB) algorithm. With the goal of limiting the exploration of risky arms, MARAB takes as arm quality its conditional value at risk. When the user-supplied risk level goes to 0, the arm quality tends toward the essential infimum of the arm distribution density, and MARAB tends toward the MIN multi-armed bandit algorithm, aimed at the arm with maximal minimal value. As a first contribution, this paper presents a theoretical analysis of the MIN algorithm under mild assumptions, establishing its robustness comparatively to UCB. The analysis is supported by extensive experimental validation of MIN and MARAB compared to UCB and state-of-art risk-aware MAB algorithms on artificial and real-world problems.