# Partially Observable Contextual Bandits with Linear Payoffs

arXiv: https://arxiv.org/abs/2409.11521 · HTML: https://mlsanity.com/paper/2409.11521/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 7 of 7)

- [Causal Bandits: Learning Good Interventions via Causal Inference](https://mlsanity.com/paper/1606.03203/) (2016)
- [Latent Bandits Revisited](https://mlsanity.com/paper/2006.08714/) (2020)
- [Latent Contextual Bandits and their Application to Personalized Recommendations for New Users](https://mlsanity.com/paper/1604.06743/) (2016)
- [Analysis of Thompson Sampling for Partially Observable Contextual Multi-Armed Bandits](https://mlsanity.com/paper/2110.12175/) (2021)
- [Thompson Sampling in Partially Observable Contextual Bandits](https://mlsanity.com/paper/2402.10289/) (2024)
- [Online learning in bandits with predicted context](https://mlsanity.com/paper/2307.13916/) (2023)
- [Efficient Algorithms for Learning to Control Bandits with Unobserved Contexts](https://mlsanity.com/paper/2202.00867/) (2022)

## Built on by (most cited first, 5 of 5)

- [Linear Bandits with Partially Observable Features](https://mlsanity.com/paper/2502.06142/) (2025)
- [COBRA: Contextual Bandit Algorithm for Ensuring Truthful Strategic Agents](https://mlsanity.com/paper/2505.23720/) (2025)
- [Scalable and Interpretable Contextual Bandits: A Literature Review and Retail Offer Prototype](https://mlsanity.com/paper/2505.16918/) (2025)
- [Learning with Incomplete Context: Linear Contextual Bandits with Pretrained Imputation](https://mlsanity.com/paper/2510.09908/) (2025)
- [Mixing Makes Markovian Contexts Cheap for Linear Bandits](https://mlsanity.com/paper/2603.12530/) (2026)

## Abstract (arXiv)

The standard contextual bandit framework assumes fully observable and actionable contexts. In this work, we consider a new bandit setting with partially observable, correlated contexts and linear payoffs, motivated by the applications in finance where decision making is based on market information that typically displays temporal correlation and is not fully observed. We make the following contributions marrying ideas from statistical signal processing with bandits: (i) We propose an algorithmic pipeline named EMKF-Bandit, which integrates system identification, filtering, and classic contextual bandit algorithms into an iterative method alternating between latent parameter estimation and decision making. (ii) We analyze EMKF-Bandit when we select Thompson sampling as the bandit algorithm and show that it incurs a sub-linear regret under conditions on filtering. (iii) We conduct numerical simulations that demonstrate the benefits and practical applicability of the proposed pipeline.