# A Simpler Alternative to Variational Regularized Counterfactual Risk Minimization

arXiv: https://arxiv.org/abs/2409.09819 · HTML: https://mlsanity.com/paper/2409.09819/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 15 Sep 2024.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation](https://mlsanity.com/paper/2008.07146/) (2020)
- [Optimal Baseline Corrections for Off-Policy Contextual Bandits](https://mlsanity.com/paper/2405.05736/) (2024)
- [Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization](https://mlsanity.com/paper/2305.01522/) (2023)
- [Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank](https://mlsanity.com/paper/2407.19943/) (2024)
- [A First Look at Selection Bias in Preference Elicitation for Recommendation](https://mlsanity.com/paper/2405.00554/) (2024)

## Abstract (arXiv)

Variance regularized counterfactual risk minimization (VRCRM) has been proposed as an alternative off-policy learning (OPL) method. VRCRM method uses a lower-bound on the f-divergence between the logging policy and the target policy as regularization during learning and was shown to improve performance over existing OPL alternatives on multi-label classification tasks. In this work, we revisit the original experimental setting of VRCRM and propose to minimize the f-divergence directly, instead of optimizing for the lower bound using a f-GAN approach. Surprisingly, we were unable to reproduce the results reported in the original setting. In response, we propose a novel simpler alternative to f-divergence optimization by minimizing a direct approximation of f-divergence directly, instead of a f-GAN based lower bound. Experiments showed that minimizing the divergence using f-GANs did not work as expected, whereas our proposed novel simpler alternative works better empirically.