# On the Unreasonable Efficiency of State Space Clustering in Personalization Tasks

arXiv: https://arxiv.org/abs/2112.13141 · HTML: https://mlsanity.com/paper/2112.13141/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 46 references that are in the map.
- First posted on arXiv 24 Dec 2021.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning](https://mlsanity.com/paper/1611.04717/) (2016)
- [Near Optimal Behavior via Approximate State Abstraction](https://mlsanity.com/paper/1701.04113/) (2017)
- [Exploration in Interactive Personalized Music Recommendation: A Reinforcement Learning Approach](https://mlsanity.com/paper/1311.6355/) (2013)
- [Learning Representations in Model-Free Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1810.10096/) (2018)
- [Data-driven unsupervised clustering of online learner behaviour](https://mlsanity.com/paper/1902.04047/) (2019)
- [Clustered Reinforcement Learning](https://mlsanity.com/paper/1906.02457/) (2019)
- [A Notation for Markov Decision Processes](https://mlsanity.com/paper/1512.09075/) (2015)

## Built on by (most cited first, 3 of 3)

- [Zero-Shot Recommendations with Pre-Trained Large Language Models for Multimodal Nudging](https://mlsanity.com/paper/2309.01026/) (2023)
- [Examining Policy Entropy of Reinforcement Learning Agents for Personalization Tasks](https://mlsanity.com/paper/2211.11869/) (2022)
- [Simulated Contextual Bandits for Personalization Tasks from Recommendation Datasets](https://mlsanity.com/paper/2210.10631/) (2022)

## Abstract (arXiv)

In this effort we consider a reinforcement learning (RL) technique for solving personalization tasks with complex reward signals. In particular, our approach is based on state space clustering with the use of a simplistic k-means algorithm as well as conventional choices of the network architectures and optimization algorithms. Numerical examples demonstrate the efficiency of different RL procedures and are used to illustrate that this technique accelerates the agent's ability to learn and does not restrict the agent's performance.