# BehaviorFormer: Discovering Behavioral Patterns via Clusterable Next-Token Prediction

arXiv: https://arxiv.org/abs/2610.16322 · HTML: https://mlsanity.com/paper/2610.16322/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 14 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 4 of 4)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Deep Clustering Using the Soft Silhouette Score: Towards Compact and Well-Separated Clusters](https://mlsanity.com/paper/2402.00608/) (2024)
- [player2vec: A Language Modeling Approach to Understand Player Behavior in Games](https://mlsanity.com/paper/2404.04234/) (2024)

## Abstract (arXiv)

Discovering behavioral patterns in open-ended games is challenging due to the lack of clear objectives or ground-truth annotations. In this work, we extend a next-token prediction Transformer with the deep clustering algorithm DCSS to learn meaningful representations with well-separated clusters. In a controlled Minecraft session with 26 players, the method discovers 23 distinct behavioral patterns that generalize across the studied players, further refining structures present in self-supervised representations into clearly separable clusters. We group the clusters into seven interpretable categories covering stationary, roaming, travel, mining, base work, logging, and farming behavior.