# Information-Gathering in Latent Bandits

arXiv: https://arxiv.org/abs/2207.03635 · HTML: https://mlsanity.com/paper/2207.03635/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 10 of 10)

- [Learning to Optimize via Information-Directed Sampling](https://mlsanity.com/paper/1403.5556/) (2014)
- [Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges](https://mlsanity.com/paper/1507.08025/) (2015)
- [Latent Bandits Revisited](https://mlsanity.com/paper/2006.08714/) (2020)
- [Learning Contextual Bandits in a Non-stationary Environment](https://mlsanity.com/paper/1805.09365/) (2018)
- [Latent Contextual Bandits and their Application to Personalized Recommendations for New Users](https://mlsanity.com/paper/1604.06743/) (2016)
- [Frappe: Understanding the Usage and Perception of Mobile App Recommendations In-The-Wild](https://mlsanity.com/paper/1505.03014/) (2015)
- [Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms](https://mlsanity.com/paper/1708.03058/) (2017)
- [Non-Stationary Latent Bandits](https://mlsanity.com/paper/2012.00386/) (2020)
- [Bayesian Policy Reuse](https://mlsanity.com/paper/1505.00284/) (2015)
- [A New Bandit Setting Balancing Information from State Evolution and Corrupted Context](https://mlsanity.com/paper/2011.07989/) (2020)

## Built on by (most cited first, 2 of 2)

- [Online learning in bandits with predicted context](https://mlsanity.com/paper/2307.13916/) (2023)
- [Beyond Random Noise: Insights on Anonymization Strategies from a Latent Bandit Study](https://mlsanity.com/paper/2310.00221/) (2023)

## Abstract (arXiv)

In the latent bandit problem, the learner has access to reward distributions and -- for the non-stationary variant -- transition models of the environment. The reward distributions are conditioned on the arm and unknown latent states. The goal is to use the reward history to identify the latent state, allowing for the optimal choice of arms in the future. The latent bandit setting lends itself to many practical applications, such as recommender and decision support systems, where rich data allows the offline estimation of environment models with online learning remaining a critical component. Previous solutions in this setting always choose the highest reward arm according to the agent's beliefs about the state, not explicitly considering the value of information-gathering arms. Such information-gathering arms do not necessarily provide the highest reward, thus may never be chosen by an agent that chooses the highest reward arms at all times. In this paper, we present a method for information-gathering in latent bandits. Given particular reward structures and transition matrices, we show that choosing the best arm given the agent's beliefs about the states incurs higher regret. Furthermore, we show that by choosing arms carefully, we obtain an improved estimation of the state distribution, and thus lower the cumulative regret through better arm choices in the future. We evaluate our method on both synthetic and real-world data sets, showing significant improvement in regret over state-of-the-art methods.