# Offline Evaluation for Reinforcement Learning-based Recommendation: A Critical Issue and Some Alternatives

arXiv: https://arxiv.org/abs/2301.00993 · HTML: https://mlsanity.com/paper/2301.00993/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2023.

## Built on by (most cited first, 3 of 3)

- [On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendation](https://mlsanity.com/paper/2307.15053/) (2023)
- [On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems](https://mlsanity.com/paper/2308.11336/) (2023)
- [A Common Misassumption in Online Experiments with Machine Learning Models](https://mlsanity.com/paper/2304.10900/) (2023)

## Abstract (arXiv)

In this paper, we argue that the paradigm commonly adopted for offline evaluation of sequential recommender systems is unsuitable for evaluating reinforcement learning-based recommenders. We find that most of the existing offline evaluation practices for reinforcement learning-based recommendation are based on a next-item prediction protocol, and detail three shortcomings of such an evaluation protocol. Notably, it cannot reflect the potential benefits that reinforcement learning (RL) is expected to bring while it hides critical deficiencies of certain offline RL agents. Our suggestions for alternative ways to evaluate RL-based recommender systems aim to shed light on the existing possibilities and inspire future research on reliable evaluation protocols.