mlsanityData to 7 Oct 2026

Bayesian Regret Minimization in Offline Bandits

arXiv 2306.01237 ↗First posted 2 Jun 2023cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study how to make decisions that minimize Bayesian regret in offline linear bandits. Prior work suggests that one must take actions with maximum lower confidence bound (LCB) on their reward. We argue that the reliance on LCB is inherently flawed in this setting and propose a new algorithm that directly minimizes upper bounds on the Bayesian regret using efficient conic optimization solvers. Our bounds build heavily on new connections to monetary risk measures. Proving a matching lower bound, we show that our upper bounds are tight, and by minimizing them we are guaranteed to outperform the LCB approach. Our numerical results on synthetic domains confirm that our approach is superior to LCB.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 48 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

13 of 48 references.

References · 13

  1. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 275
  2. 2021Bellman-consistent Pessimism for Offline Reinforcement Learningcited by 267
  3. 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 160
  4. 2019A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Normcited by 105
  5. 2019Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPscited by 37
  6. 2022JuMP 1.0: Recent improvements to a modeling language for mathematical optimizationcited by 34
  7. 2020Bayesian Robust Optimization for Imitation Learningcited by 28
  8. 2021On the Optimality of Batch Policy Optimization Algorithmscited by 28
  9. 2021Policy Gradient Bayesian Robust Optimization for Imitation Learningcited by 11
  10. 2022ARMOR: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Datacited by 9
  11. 2022Multi-Task Off-Policy Learning from Bandit Feedbackcited by 9
  12. 2020Soft-Robust Algorithms for Batch Reinforcement Learningcited by 3
1 more
  1. 2016An Extension of Chubanov's Polynomial-Time Linear Programming Algorithm to Second-Order Cone Programmingcited by 1

Built on by

Not cited yet.