# Risk-Averse Multi-Armed Bandits with Unobserved Confounders: A Case Study in Emotion Regulation in Mobile Health

arXiv: https://arxiv.org/abs/2209.04356 · HTML: https://mlsanity.com/paper/2209.04356/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 31 references that are in the map.
- First posted on arXiv 9 Sep 2022.

## Builds on (most cited first, 7 of 7)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Probabilities of Causation: Bounds and Identification](https://mlsanity.com/paper/1301.3898/) (2013)
- [Bounds on Causal Effects and Application to High Dimensional Data](https://mlsanity.com/paper/2106.12121/) (2021)
- [Risk-Averse Multi-Armed Bandit Problems under Mean-Variance Measure](https://mlsanity.com/paper/1604.05257/) (2016)
- [Offline Contextual Multi-armed Bandits for Mobile Health Interventions: A Case Study on Emotion Regulation](https://mlsanity.com/paper/2008.09472/) (2020)
- [Learning without Knowing: Unobserved Context in Continuous Transfer Reinforcement Learning](https://mlsanity.com/paper/2106.03833/) (2021)
- [Transfer Reinforcement Learning under Unobserved Contextual Information](https://mlsanity.com/paper/2003.04427/) (2020)

## Built on by (most cited first, 2 of 2)

- [Time-uniform confidence bands for the CDF under nonstationarity](https://mlsanity.com/paper/2302.14248/) (2023)
- [Risk-Averse Best Arm Set Identification with Fixed Budget and Fixed Confidence](https://mlsanity.com/paper/2506.22253/) (2025)

## Abstract (arXiv)

In this paper, we consider a risk-averse multi-armed bandit (MAB) problem where the goal is to learn a policy that minimizes the risk of low expected return, as opposed to maximizing the expected return itself, which is the objective in the usual approach to risk-neutral MAB. Specifically, we formulate this problem as a transfer learning problem between an expert and a learner agent in the presence of contexts that are only observable by the expert but not by the learner. Thus, such contexts are unobserved confounders (UCs) from the learner's perspective. Given a dataset generated by the expert that excludes the UCs, the goal for the learner is to identify the true minimum-risk arm with fewer online learning steps, while avoiding possible biased decisions due to the presence of UCs in the expert's data.