# Bandits with Partially Observable Confounded Data

arXiv: https://arxiv.org/abs/2006.06731 · HTML: https://mlsanity.com/paper/2006.06731/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 22 of 22)

- [A Survey on Causal Reinforcement Learning](https://mlsanity.com/paper/2302.05209/) (2023)
- [Leveraging Offline Data in Online Reinforcement Learning](https://mlsanity.com/paper/2211.04974/) (2022)
- [Causal Deep Learning](https://mlsanity.com/paper/2303.02186/) (2023)
- [Invariant Policy Learning: A Causal Perspective](https://mlsanity.com/paper/2106.00808/) (2021)
- [On Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning](https://mlsanity.com/paper/2110.06539/) (2021)
- [Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounders](https://mlsanity.com/paper/2302.00662/) (2023)
- [Analysis of Thompson Sampling for Partially Observable Contextual Multi-Armed Bandits](https://mlsanity.com/paper/2110.12175/) (2021)
- [Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding](https://mlsanity.com/paper/2306.01157/) (2023)
- [Online Decision Mediation](https://mlsanity.com/paper/2310.18601/) (2023)
- [Linear Bandits with Partially Observable Features](https://mlsanity.com/paper/2502.06142/) (2025)
- [Efficient Algorithms for Learning to Control Bandits with Unobserved Contexts](https://mlsanity.com/paper/2202.00867/) (2022)
- [Worst-case Performance of Greedy Policies in Bandits with Imperfect Context Observations](https://mlsanity.com/paper/2204.04773/) (2022)
- [A Unified Framework of Policy Learning for Contextual Bandit with Confounding Bias and Missing Observations](https://mlsanity.com/paper/2303.11187/) (2023)
- [Predictive Performance Comparison of Decision Policies Under Confounding](https://mlsanity.com/paper/2404.00848/) (2024)
- [Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach](https://mlsanity.com/paper/2312.12731/) (2023)
- [Benchmarks for Reinforcement Learning with Biased Offline Data and Imperfect Simulators](https://mlsanity.com/paper/2407.00806/) (2024)
- [Contextual Bandit with Herding Effects: Algorithms and Recommendation Applications](https://mlsanity.com/paper/2408.14432/) (2024)
- [Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits](https://mlsanity.com/paper/2604.24016/) (2026)
- [Dual Instrumental Method for Confounded Kernelized Bandits](https://mlsanity.com/paper/2209.03224/) (2022)
- [Ranking with Popularity Bias: User Welfare under Self-Amplification Dynamics](https://mlsanity.com/paper/2305.18333/) (2023)
- [Deconfounded Warm-Start Thompson Sampling with Applications to Precision Medicine](https://mlsanity.com/paper/2505.17283/) (2025)
- [Target-Dependent Limits of Causal Repair: A Leading-Log Frontier in a Gaussian Model](https://mlsanity.com/paper/2610.00424/) (2026)

## Abstract (arXiv)

We study linear contextual bandits with access to a large, confounded, offline dataset that was sampled from some fixed policy. We show that this problem is closely related to a variant of the bandit problem with side information. We construct a linear bandit algorithm that takes advantage of the projected information, and prove regret bounds. Our results demonstrate the ability to take advantage of confounded offline data. Particularly, we prove regret bounds that improve current bounds by a factor related to the visible dimensionality of the contexts in the data. Our results indicate that confounded offline data can significantly improve online learning algorithms. Finally, we demonstrate various characteristics of our approach through synthetic simulations.