mlsanityData to 14 Oct 2026

Monte Carlo Rollout Policy for Recommendation Systems with Dynamic User Behavior

arXiv 2102.04321 ↗First posted 8 Feb 2021eess.SY cs.LG cs.SYcited by 1

Not on the map. Why?

Abstract (arXiv)

We model online recommendation systems using the hidden Markov multi-state restless multi-armed bandit problem. To solve this we present Monte Carlo rollout policy. We illustrate numerically that Monte Carlo rollout policy performs better than myopic policy for arbitrary transition dynamics with no specific structure. But, when some structure is imposed on the transition dynamics, myopic policy performs better than Monte Carlo rollout policy.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 2 of its 21 references that are in the map. First posted on arXiv 8 Feb 2021.

Builds on

2 of 21 references.

References · 2

  1. 2016On the Whittle Index for Restless Multi-armed Hidden Markov Banditscited by 12
  2. 2020Simulation Based Algorithms for Markov Decision Processes and Multi-Action Restless Banditscited by 7

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Lagrangian Relaxation for Multi-Action Partially Observable Restless Bandits: Heuristic Policies and Indexabilitycited by 0