# player2vec: A Language Modeling Approach to Understand Player Behavior in Games

arXiv: https://arxiv.org/abs/2404.04234 · HTML: https://mlsanity.com/paper/2404.04234/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation](https://mlsanity.com/paper/1812.02646/) (2018)
- [ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation](https://mlsanity.com/paper/1711.06632/) (2017)
- [Using Temporal Data for Making Recommendations](https://mlsanity.com/paper/1301.2320/) (2013)

## Built on by (most cited first, 5 of 5)

- [Can Large Language Models Capture Video Game Engagement?](https://mlsanity.com/paper/2502.04379/) (2025)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [Behavior Structformer: Learning Players Representations with Structured Tokenization](https://mlsanity.com/paper/2406.05274/) (2024)
- [AI for Handball: predicting and explaining the 2024 Olympic Games tournament with Deep Learning and Large Language Models](https://mlsanity.com/paper/2407.15987/) (2024)
- [BehaviorFormer: Discovering Behavioral Patterns via Clusterable Next-Token Prediction](https://mlsanity.com/paper/2610.16322/) (2026)

## Abstract (arXiv)

Methods for learning latent user representations from historical behavior logs have gained traction for recommendation tasks in e-commerce, content streaming, and other settings. However, this area still remains relatively underexplored in video and mobile gaming contexts. In this work, we present a novel method for overcoming this limitation by extending a long-range Transformer model from the natural language processing domain to player behavior data. We discuss specifics of behavior tracking in games and propose preprocessing and tokenization approaches by viewing in-game events in an analogous way to words in sentences, thus enabling learning player representations in a self-supervised manner in the absence of ground-truth annotations. We experimentally demonstrate the efficacy of the proposed approach in fitting the distribution of behavior events by evaluating intrinsic language modeling metrics. Furthermore, we qualitatively analyze the emerging structure of the learned embedding space and show its value for generating insights into behavior patterns to inform downstream applications.