# Behavior Structformer: Learning Players Representations with Structured Tokenization

arXiv: https://arxiv.org/abs/2406.05274 · HTML: https://mlsanity.com/paper/2406.05274/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Modeling Users' Behavior Sequences with Hierarchical Explainable Network for Cross-domain Fraud Detection](https://mlsanity.com/paper/2201.01004/) (2022)
- [Sequential Behavioral Data Processing Using Deep Learning and the Markov Transition Field in Online Fraud Detection](https://mlsanity.com/paper/1808.05329/) (2018)
- [player2vec: A Language Modeling Approach to Understand Player Behavior in Games](https://mlsanity.com/paper/2404.04234/) (2024)

## Built on by (most cited first, 1 of 1)

- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)

## Abstract (arXiv)

In this paper, we introduce the Behavior Structformer, a method for modeling user behavior using structured tokenization within a Transformer-based architecture. By converting tracking events into dense tokens, this approach enhances model training efficiency and effectiveness. We demonstrate its superior performance through ablation studies and benchmarking against traditional tabular and semi-structured baselines. The results indicate that structured tokenization with sequential processing significantly improves behavior modeling.