mlsanityData to 14 Oct 2026

Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning

arXiv 2405.14335 ↗First posted 23 May 2024stat.ML cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

This work investigates the offline formulation of the contextual bandit problem, where the goal is to leverage past interactions collected under a behavior policy to evaluate, select, and learn new, potentially better-performing, policies. Motivated by critical applications, we move beyond point estimators. Instead, we adopt the principle of pessimism where we construct upper bounds that assess a policy's worst-case performance, enabling us to confidently select and learn improved policies. Precisely, we introduce novel, fully empirical concentration bounds for a broad class of importance weighting risk estimators. These bounds are general enough to cover most existing estimators and pave the way for the development of new ones. In particular, our pursuit of the tightest bound within this class motivates a novel estimator (LS), that logarithmically smooths large importance weights. The bound for LS is provably tighter than its competitors, and naturally results in improved policy selection and learning strategies. Extensive policy evaluation, selection, and learning experiments highlight the versatility and favorable performance of LS.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 33 of its 63 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

33 of 63 references.

References · 33

  1. 2020Is Pessimism Provably Efficient for Offline RL?cited by 364
  2. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  3. 2018Top-K Off-Policy Correction for a REINFORCE Recommender Systemcited by 182
  4. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  5. 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
  6. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  7. 2014Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)cited by 109
  8. 2016Off-policy evaluation for slate recommendationcited by 103
  9. 2018Offline A/B testing for Recommender Systemscited by 96
  10. 2019Doubly robust off-policy evaluation with shrinkagecited by 87
  11. 2020CoinDICE: Off-Policy Confidence Interval Estimationcited by 64
  12. 2020Off-policy Bandits with Deficient Supportcited by 61
21 more
  1. 2018Policy Evaluation and Optimization with Continuous Treatmentscited by 58
  2. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 54
  3. 2019Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adaptingcited by 42
  4. 2020Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weightingcited by 38
  5. 2023Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modelingcited by 34
  6. 2018Bayesian Counterfactual Risk Minimizationcited by 28
  7. 2019Semi-Parametric Efficient Policy Learning with Continuous Actionscited by 26
  8. 2022PAC-Bayesian Offline Contextual Bandits With Guaranteescited by 20
  9. 2020BLOB : A Probabilistic Model for Recommendation that Combines Organic and Bandit Signalscited by 19
  10. 2023Exponential Smoothing for Off-Policy Learningcited by 15
  11. 2023Oracle-Efficient Pessimism: Offline Policy Optimization in Contextual Banditscited by 15
  12. 2023Importance-Weighted Offline Learning Done Rightcited by 15
  13. 2023Off-Policy Evaluation for Large Action Spaces via Policy Convolutioncited by 14
  14. 2023Marginal Density Ratio for Off-Policy Evaluation in Contextual Banditscited by 9
  15. 2022Probabilistic Rank and Reward: A Scalable Model for Slate Recommendationcited by 8
  16. 2022PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparisoncited by 7
  17. 2023Learning Action Embeddings for Off-Policy Evaluationcited by 7
  18. 2020Improving Offline Contextual Bandits with Distributional Robustnesscited by 5
  19. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  20. 2022Fast Offline Policy Optimization for Large Scale Recommendationcited by 4
  21. 2023Fast Slate Policy Optimization: Going Beyond Plackett-Lucecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 5.

16 papers.

Sort:

By area:

  1. 2025Towards a Sharp Analysis of Offline Policy Learning for f-Divergence-Regularized Contextual Banditscited by 10
  2. 2025Heavy-Tailed Linear Bandits: Huber Regression with One-Pass Updatecited by 5
  3. 2025Improved Offline Contextual Bandits with Second-Order Bounds: Betting and Freezingcited by 4
  4. 2025Exploiting Similarities in A/B Testing with Off-Policy Estimationcited by 2
  5. 2025A General Framework for Off-Policy Learning with Partially-Observed Rewardcited by 1
  6. 2025Non-Linear Counterfactual Aggregate Optimizationcited by 1
  7. 2025Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimationcited by 1
  8. 2025Offline Contextual Bandit with Counterfactual Sample Identificationcited by 1
  9. 2026A More Accurate Algorithm Comparison through A/B Testing using Offline Evaluation Methodscited by 1
  10. 2025Clustered KL-barycenter design for policy evaluationcited by 0
  11. 2025Sequential Off-Policy Learning with Logarithmic Smoothingcited by 0
  12. 2025Evaluating and Learning Robust Bandit Policies Under Uncertain Causal Mechanismscited by 0
4 more
  1. 2026Kernel weighted importance sampling for off-policy evaluation in contextual banditscited by 0
  2. 2026Evaluating covariate balance for long time horizon Markov decision processescited by 0
  3. 2026Cross-Domain Off-Policy Evaluation and Learning for Contextual Banditscited by 0
  4. 2026TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMscited by 0