# Analysis of Thompson Sampling for Partially Observable Contextual Multi-Armed Bandits

arXiv: https://arxiv.org/abs/2110.12175 · HTML: https://mlsanity.com/paper/2110.12175/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 23 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Learning to Optimize Via Posterior Sampling](https://mlsanity.com/paper/1301.2609/) (2013)
- [Mostly Exploration-Free Algorithms for Contextual Bandits](https://mlsanity.com/paper/1704.09011/) (2017)
- [Latent Bandits Revisited](https://mlsanity.com/paper/2006.08714/) (2020)
- [Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes](https://mlsanity.com/paper/1909.02553/) (2019)
- [Bandits with Partially Observable Confounded Data](https://mlsanity.com/paper/2006.06731/) (2020)
- [On Adaptive Linear-Quadratic Regulators](https://mlsanity.com/paper/1806.10749/) (2018)
- [Greedy Algorithm almost Dominates in Smoothed Contextual Bandits](https://mlsanity.com/paper/2005.10624/) (2020)
- [Latent Contextual Bandits and their Application to Personalized Recommendations for New Users](https://mlsanity.com/paper/1604.06743/) (2016)
- [Context Attentive Bandits: Contextual Bandit with Restricted Context](https://mlsanity.com/paper/1705.03821/) (2017)
- [Optimism-Based Adaptive Regulation of Linear-Quadratic Systems](https://mlsanity.com/paper/1711.07230/) (2017)
- [Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments](https://mlsanity.com/paper/2005.13857/) (2020)
- [Note on Thompson sampling for large decision problems](https://mlsanity.com/paper/1905.04735/) (2019)

## Built on by (most cited first, 10 of 10)

- [Thompson Sampling in Partially Observable Contextual Bandits](https://mlsanity.com/paper/2402.10289/) (2024)
- [Partially Observable Contextual Bandits with Linear Payoffs](https://mlsanity.com/paper/2409.11521/) (2024)
- [Online Decision Mediation](https://mlsanity.com/paper/2310.18601/) (2023)
- [Efficient Algorithms for Learning to Control Bandits with Unobserved Contexts](https://mlsanity.com/paper/2202.00867/) (2022)
- [Remote Contextual Bandits](https://mlsanity.com/paper/2202.05182/) (2022)
- [Worst-case Performance of Greedy Policies in Bandits with Imperfect Context Observations](https://mlsanity.com/paper/2204.04773/) (2022)
- [Follow-ups Also Matter: Improving Contextual Bandits via Post-serving Contexts](https://mlsanity.com/paper/2309.13896/) (2023)
- [Distributed Multi-Task Learning for Stochastic Bandits with Context Distribution and Stage-wise Constraints](https://mlsanity.com/paper/2401.11563/) (2024)
- [Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis](https://mlsanity.com/paper/2401.11565/) (2024)
- [Robust Linear Dueling Bandits with Post-serving Context under Unknown Delays and Adversarial Corruptions](https://mlsanity.com/paper/2605.01752/) (2026)

## Abstract (arXiv)

Contextual multi-armed bandits are classical models in reinforcement learning for sequential decision-making associated with individual information. A widely-used policy for bandits is Thompson Sampling, where samples from a data-driven probabilistic belief about unknown parameters are used to select the control actions. For this computationally fast algorithm, performance analyses are available under full context-observations. However, little is known for problems that contexts are not fully observed. We propose a Thompson Sampling algorithm for partially observable contextual multi-armed bandits, and establish theoretical performance guarantees. Technically, we show that the regret of the presented policy scales logarithmically with time and the number of arms, and linearly with the dimension. Further, we establish rates of learning unknown parameters, and provide illustrative numerical analyses.