# Clustered Reinforcement Learning

arXiv: https://arxiv.org/abs/1906.02457 · HTML: https://mlsanity.com/paper/1906.02457/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2019.

## Built on by (most cited first, 5 of 5)

- [On the Unreasonable Efficiency of State Space Clustering in Personalization Tasks](https://mlsanity.com/paper/2112.13141/) (2021)
- [Goal-conditioned Offline Reinforcement Learning through State Space Partitioning](https://mlsanity.com/paper/2303.09367/) (2023)
- [Just Cluster It: An Approach for Exploration in High-Dimensions using Clustering and Pre-Trained Representations](https://mlsanity.com/paper/2402.03138/) (2024)
- [Dataset Clustering for Improved Offline Policy Learning](https://mlsanity.com/paper/2402.09550/) (2024)
- [Data-dependent Exploration for Online Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2605.04477/) (2026)

## Abstract (arXiv)

Exploration strategy design is one of the challenging problems in reinforcement learning~(RL), especially when the environment contains a large state space or sparse rewards. During exploration, the agent tries to discover novel areas or high reward~(quality) areas. In most existing methods, the novelty and quality in the neighboring area of the current state are not well utilized to guide the exploration of the agent. To tackle this problem, we propose a novel RL framework, called underlineclustered underlinereinforcement underlinelearning~(CRL), for efficient exploration in RL. CRL adopts clustering to divide the collected states into several clusters, based on which a bonus reward reflecting both novelty and quality in the neighboring area~(cluster) of the current state is given to the agent. Experiments on a continuous control task and several Atari 2600 games show that CRL can outperform other state-of-the-art methods to achieve the best performance in most cases.