# Distributionally Robust Batch Contextual Bandits

arXiv: https://arxiv.org/abs/2006.05630 · HTML: https://mlsanity.com/paper/2006.05630/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 20 of 20)

- [Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coverage](https://mlsanity.com/paper/2305.09659/) (2023)
- [Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2202.09667/) (2022)
- [Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaR](https://mlsanity.com/paper/2302.03201/) (2023)
- [Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithms](https://mlsanity.com/paper/2404.03578/) (2024)
- [Robust Bandit Learning with Imperfect Context](https://mlsanity.com/paper/2102.05018/) (2021)
- [Distributionally Robust Policy Learning under Concept Drifts](https://mlsanity.com/paper/2412.14297/) (2024)
- [ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learning](https://mlsanity.com/paper/2508.03768/) (2025)
- [Best Arm Identification with Possibly Biased Offline Data](https://mlsanity.com/paper/2505.23165/) (2025)
- [Adversarial Training for Gradient Descent: Analysis Through its Continuous-time Approximation](https://mlsanity.com/paper/2105.08037/) (2021)
- [STEEL: Singularity-aware Reinforcement Learning](https://mlsanity.com/paper/2301.13152/) (2023)
- [Towards Optimal Offline Reinforcement Learning](https://mlsanity.com/paper/2503.12283/) (2025)
- [Towards Robust Off-policy Learning for Runtime Uncertainty](https://mlsanity.com/paper/2202.13337/) (2022)
- [Risk-Aware Linear Bandits: Theory and Applications in Smart Order Routing](https://mlsanity.com/paper/2208.02389/) (2022)
- [Towards Domain Adaptive Neural Contextual Bandits](https://mlsanity.com/paper/2406.09564/) (2024)
- [Online Bayesian Risk-Averse Reinforcement Learning](https://mlsanity.com/paper/2509.14077/) (2025)
- [Towards Robust Off-Policy Evaluation via Human Inputs](https://mlsanity.com/paper/2209.08682/) (2022)
- [Robust Predictive Modeling Under Unseen Data Distribution Shifts: A Methodological Commentary](https://mlsanity.com/paper/2503.03399/) (2025)
- [Duality and Policy Evaluation in Distributionally Robust Bayesian Diffusion Control](https://mlsanity.com/paper/2506.19294/) (2025)
- [Evaluating and Learning Robust Bandit Policies Under Uncertain Causal Mechanisms](https://mlsanity.com/paper/2508.02812/) (2025)
- [Wasserstein Policy Learning for Distributional Outcomes](https://mlsanity.com/paper/2606.19117/) (2026)

## Abstract (arXiv)

Policy learning using historical observational data is an important problem that has found widespread applications. Examples include selecting offers, prices, advertisements to send to customers, as well as selecting which medication to prescribe to a patient. However, existing literature rests on the crucial assumption that the future environment where the learned policy will be deployed is the same as the past environment that has generated the data -- an assumption that is often false or too coarse an approximation. In this paper, we lift this assumption and aim to learn a distributionally robust policy with incomplete observational data. We first present a policy evaluation procedure that allows us to assess how well the policy does under the worst-case environment shift. We then establish a central limit theorem type guarantee for this proposed policy evaluation scheme. Leveraging this evaluation scheme, we further propose a novel learning algorithm that is able to learn a policy that is robust to adversarial perturbations and unknown covariate shifts with a performance guarantee based on the theory of uniform convergence. Finally, we empirically test the effectiveness of our proposed algorithm in synthetic datasets and demonstrate that it provides the robustness that is missing using standard policy learning algorithms. We conclude the paper by providing a comprehensive application of our methods in the context of a real-world voting dataset.