# Offline Evaluation of Reward-Optimizing Recommender Systems: The Case of Simulation

arXiv: https://arxiv.org/abs/2209.08642 · HTML: https://mlsanity.com/paper/2209.08642/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 18 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Offline A/B testing for Recommender Systems](https://mlsanity.com/paper/1801.07030/) (2018)
- [RecSim: A Configurable Simulation Platform for Recommender Systems](https://mlsanity.com/paper/1909.04847/) (2019)
- [RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising](https://mlsanity.com/paper/1808.00720/) (2018)
- [A Critical Study on Data Leakage in Recommender System Offline Evaluation](https://mlsanity.com/paper/2010.11060/) (2020)
- [Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions](https://mlsanity.com/paper/2007.12986/) (2020)
- [Offline Evaluation of Ranking Policies with Click Models](https://mlsanity.com/paper/1804.10488/) (2018)
- [Carousel Personalization in Music Streaming Apps with Contextual Bandits](https://mlsanity.com/paper/2009.06546/) (2020)
- [BLOB : A Probabilistic Model for Recommendation that Combines Organic and Bandit Signals](https://mlsanity.com/paper/2008.12504/) (2020)
- [Combining Reward and Rank Signals for Slate Recommendation](https://mlsanity.com/paper/2107.12455/) (2021)
- [Three Methods for Training on Bandit Feedback](https://mlsanity.com/paper/1904.10799/) (2019)
- [On the Value of Bandit Feedback for Offline Recommender System Evaluation](https://mlsanity.com/paper/1907.12384/) (2019)
- [Learning from Bandit Feedback: An Overview of the State-of-the-art](https://mlsanity.com/paper/1909.08471/) (2019)

## Built on by (most cited first, 2 of 2)

- [Identifying Offline Metrics that Predict Online Impact: A Pragmatic Strategy for Real-World Recommender Systems](https://mlsanity.com/paper/2507.09566/) (2025)
- [On-Policy and Off-Policy Learning for Large Action Spaces](https://mlsanity.com/paper/2607.28408/) (2026)

## Abstract (arXiv)

Both in academic and industry-based research, online evaluation methods are seen as the golden standard for interactive applications like recommendation systems. Naturally, the reason for this is that we can directly measure utility metrics that rely on interventions, being the recommendations that are being shown to users. Nevertheless, online evaluation methods are costly for a number of reasons, and a clear need remains for reliable offline evaluation procedures. In industry, offline metrics are often used as a first-line evaluation to generate promising candidate models to evaluate online. In academic work, limited access to online systems makes offline metrics the de facto approach to validating novel methods. Two classes of offline metrics exist: proxy-based methods, and counterfactual methods. The first class is often poorly correlated with the online metrics we care about, and the latter class only provides theoretical guarantees under assumptions that cannot be fulfilled in real-world environments. Here, we make the case that simulation-based comparisons provide ways forward beyond offline metrics, and argue that they are a preferable means of evaluation.