mlsanityData to 7 Oct 2026

Sequential Counterfactual Risk Minimization

arXiv 2302.12120 ↗First posted 23 Feb 2023cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Counterfactual Risk Minimization (CRM) is a framework for dealing with the logged bandit feedback problem, where the goal is to improve a logging policy using offline data. In this paper, we explore the case where it is possible to deploy learned policies multiple times and acquire new data. We extend the CRM principle and its theory to this scenario, which we call "Sequential Counterfactual Risk Minimization (SCRM)." We introduce a novel counterfactual estimator and identify conditions that can improve the performance of CRM in terms of excess risk and regret rates, by using an analysis similar to restart strategies in accelerated optimization methods. We also provide an empirical evaluation of our method in both discrete and continuous action settings, and demonstrate the benefits of multiple deployments of CRM.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 58 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

24 of 58 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2021Convex optimizationcited by 2,479
  4. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 260
  5. 2020Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oraclescited by 195
  6. 2013Finite-Time Analysis of Kernelised Contextual Banditscited by 177
  7. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  8. 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 114
  9. 2019Batched Multi-armed Bandits Problemcited by 97
  10. 2020CoinDICE: Off-Policy Confidence Interval Estimationcited by 64
  11. 2020Sequential Batch Learning in Finite-Action Linear Contextual Banditscited by 63
  12. 2020Off-policy Bandits with Deficient Supportcited by 60
12 more
  1. 2018Policy Evaluation and Optimization with Continuous Treatmentscited by 57
  2. 2016Q(λ) with Off-Policy Correctionscited by 48
  3. 2021Acceleration Methodscited by 48
  4. 2015Fast rates in statistical and online learningcited by 47
  5. 2016MetaGrad: Multiple Learning Rates in Online Learningcited by 46
  6. 2019Semi-Parametric Efficient Policy Learning with Continuous Actionscited by 26
  7. 2019Distributionally Robust Counterfactual Risk Minimizationcited by 26
  8. 2021Policy Learning with Adaptively Collected Datacited by 21
  9. 2020Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsificationcited by 13
  10. 2021Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learningcited by 10
  11. 2018Efficient online algorithms for fast-rate regret bounds under sparsitycited by 8
  12. 2020Counterfactual Learning of Stochastic Policies with Continuous Actionscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Improved Offline Contextual Bandits with Second-Order Bounds: Betting and Freezingcited by 3
  2. 2025Kernel Treatment Effects with Adaptively Collected Datacited by 1
  3. 2025Learning Counterfactual Outcomes Under Rank Preservationcited by 0
  4. 2025Sequential Off-Policy Learning with Logarithmic Smoothingcited by 0
  5. 2025Fast Best-in-Class Regret for Contextual Banditscited by 0
  6. 2026CAPER: Clause-Aligned Process Supervision for Text-to-SQLcited by 0