mlsanityData to 14 Oct 2026

Exponential Smoothing for Off-Policy Learning

arXiv 2305.15877 ↗First posted 25 May 2023cs.LG cs.AI stat.MLcited by 15

Not on the map. Why?

Abstract (arXiv)

Off-policy learning (OPL) aims at finding improved policies from logged bandit data, often by minimizing the inverse propensity scoring (IPS) estimator of the risk. In this work, we investigate a smooth regularization for IPS, for which we derive a two-sided PAC-Bayes generalization bound. The bound is tractable, scalable, interpretable and provides learning certificates. In particular, it is also valid for standard IPS without making the assumption that the importance weights are bounded. We demonstrate the relevance of our approach and its favorable performance through a set of learning tasks. Since our bound holds for standard IPS, we are able to provide insight into when regularizing IPS is useful. Namely, we identify cases where regularization might not be needed. This goes against the belief that, in practice, clipped IPS often enjoys favorable performance than standard IPS in OPL.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 30 of its 66 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

30 of 66 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  4. 2017A Tutorial on Thompson Samplingcited by 433
  5. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  6. 2014Taming the Monster: A Fast and Simple Algorithm for Contextual Banditscited by 201
  7. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  8. 2014A* Samplingcited by 151
  9. 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
  10. 2019A Primer on PAC-Bayesian Learningcited by 132
  11. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  12. 2018Offline A/B testing for Recommender Systemscited by 96
18 more
  1. 2019Doubly robust off-policy evaluation with shrinkagecited by 87
  2. 2020Off-policy Bandits with Deficient Supportcited by 61
  3. 2022Off-Policy Evaluation for Large Action Spaces via Embeddingscited by 54
  4. 2016Cascading Bandits for Large-Scale Recommendation Problemscited by 45
  5. 2020Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weightingcited by 38
  6. 2022Contextual Bandits with Large Action Spaces: Made Practicalcited by 33
  7. 2020Optimal Off-Policy Evaluation from Multiple Logging Policiescited by 30
  8. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 30
  9. 2018Bayesian Counterfactual Risk Minimizationcited by 28
  10. 2019Distributionally Robust Counterfactual Risk Minimizationcited by 26
  11. 2022PAC-Bayesian Offline Contextual Bandits With Guaranteescited by 20
  12. 2020BLOB : A Probabilistic Model for Recommendation that Combines Organic and Bandit Signalscited by 19
  13. 2022Deep Hierarchy in Banditscited by 19
  14. 2019Efron-Stein PAC-Bayesian Inequalitiescited by 16
  15. 2022Mixed-Effect Thompson Samplingcited by 14
  16. 2021Adaptive Importance Sampling meets Mirror Descent: a Bias-variance tradeoffcited by 13
  17. 2022Probabilistic Rank and Reward: A Scalable Model for Slate Recommendationcited by 8
  18. 2021Combining Reward and Rank Signals for Slate Recommendationcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 7; 2025: 6.

15 papers.

Sort:

By area:

  1. 2024Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learningcited by 16
  2. 2023Oracle-Efficient Pessimism: Offline Policy Optimization in Contextual Banditscited by 15
  3. 2024Long-term Off-Policy Evaluation and Learningcited by 9
  4. 2024Diffusion Models Meet Contextual Banditscited by 5
  5. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  6. 2025Improved Offline Contextual Bandits with Second-Order Bounds: Betting and Freezingcited by 4
  7. 2023Fast Slate Policy Optimization: Going Beyond Plackett-Lucecited by 3
  8. 2024Cross-Validated Off-Policy Evaluationcited by 3
  9. 2024Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Samplingcited by 3
  10. 2025Exploiting Similarities in A/B Testing with Off-Policy Estimationcited by 2
  11. 2025Non-Linear Counterfactual Aggregate Optimizationcited by 1
  12. 2025Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimationcited by 1
3 more
  1. 2025Offline Contextual Bandit with Counterfactual Sample Identificationcited by 1
  2. 2024Position Paper: Why the Shooting in the Dark Method Dominates Recommender Systems Practice; A Call to Abandon Anti-Utopian Thinkingcited by 0
  3. 2025Sequential Off-Policy Learning with Logarithmic Smoothingcited by 0