# Monte Carlo Rollout Policy for Recommendation Systems with Dynamic User Behavior

arXiv: https://arxiv.org/abs/2102.04321 · HTML: https://mlsanity.com/paper/2102.04321/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 21 references that are in the map.
- First posted on arXiv 8 Feb 2021.

## Builds on (most cited first, 2 of 2)

- [On the Whittle Index for Restless Multi-armed Hidden Markov Bandits](https://mlsanity.com/paper/1603.04739/) (2016)
- [Simulation Based Algorithms for Markov Decision Processes and Multi-Action Restless Bandits](https://mlsanity.com/paper/2007.12933/) (2020)

## Built on by (most cited first, 1 of 1)

- [Lagrangian Relaxation for Multi-Action Partially Observable Restless Bandits: Heuristic Policies and Indexability](https://mlsanity.com/paper/2509.00415/) (2025)

## Abstract (arXiv)

We model online recommendation systems using the hidden Markov multi-state restless multi-armed bandit problem. To solve this we present Monte Carlo rollout policy. We illustrate numerically that Monte Carlo rollout policy performs better than myopic policy for arbitrary transition dynamics with no specific structure. But, when some structure is imposed on the transition dynamics, myopic policy performs better than Monte Carlo rollout policy.