mlsanityData to 14 Oct 2026

Pessimistic Risk-Aware Policy Learning in Contextual Bandits

arXiv 2605.15620 ↗First posted 15 May 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study risk-aware offline policy learning, aiming to learn a decision rule from logged data that is optimal under general risk criteria. This problem is crucial in high-stakes domains where online interaction is infeasible and adverse outcomes must be carefully controlled. However, existing literature on offline contextual bandits either centers on expected-reward criteria or restricts risk considerations to policy evaluation instead of optimization. In this work, we propose a unified distributional framework for optimizing Lipschitz-continuous risk functionals, a broad class of risk measures encompassing mean-variance, entropic risk, and conditional value-at-risk, among others. By developing novel empirical concentration inequalities for importance sampling-based distributional estimators, our analysis derives data-dependent suboptimality bounds with an tildeO(1/√n) rate, without relying on restrictive uniform overlap assumptions. This rate is minimax optimal and matches that of risk-neutral offline policy optimization, indicating that optimizing general Lipschitz risk criteria incurs no additional statistical cost relative to the expected-reward.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 49 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

27 of 49 references.

References · 27

  1. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  2. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
  3. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  4. 2020Is Pessimism Provably Efficient for Offline RL?cited by 364
  5. 2014From Predictive to Prescriptive Analyticscited by 143
  6. 2020The Importance of Pessimism in Fixed-Dataset Policy Optimizationcited by 99
  7. 2021Conservative Offline Distributional Reinforcement Learningcited by 79
  8. 2018Offline Multi-Action Policy Learning: Generalization and Optimizationcited by 65
  9. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  10. 2019Being Optimistic to Be Conservative: Quickly Learning a CVaR Policycited by 63
  11. 2020Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regretcited by 58
  12. 2017Balanced Policy Evaluation and Learningcited by 52
15 more
  1. 2021Universal Off-Policy Evaluationcited by 43
  2. 2015Cumulative Prospect Theory Meets Reinforcement Learning: Prediction and Controlcited by 34
  3. 2022Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequalitycited by 32
  4. 2023Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaRcited by 32
  5. 2021Off-Policy Risk Assessment in Contextual Banditscited by 29
  6. 2017Nonparametric causal effects based on incremental propensity score interventionscited by 28
  7. 2022Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 26
  8. 2022One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learningcited by 23
  9. 2021Policy Learning with Adaptively Collected Datacited by 21
  10. 2019A Wasserstein distance approach for concentration of empirical risk estimatescited by 18
  11. 2022Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Boundscited by 16
  12. 2022Upper bounds on the Natarajan dimensions of some function classescited by 8
  13. 2022Off-Policy Risk Assessment in Markov Decision Processescited by 7
  14. 2023Positivity-free Policy Learning with Observational Datacited by 6
  15. 2024Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learningcited by 6

Built on by

Not cited yet.