# General-Purpose User Modeling with Behavioral Logs: A Snapchat Case Study

arXiv: https://arxiv.org/abs/2312.12111 · HTML: https://mlsanity.com/paper/2312.12111/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2023.

## Built on by (most cited first, 4 of 4)

- [Revisiting Self-attention for Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2505.21811/) (2025)
- [Integrating Sequential and Relational Modeling for User Events: Datasets and Prediction Tasks](https://mlsanity.com/paper/2510.11903/) (2025)
- [Abacus: Self-Supervised Event Counting-Aligned Distributional Pretraining for Sequential User Modeling](https://mlsanity.com/paper/2512.16581/) (2025)
- [Log2Motion: Biomechanical Motion Synthesis from Touch Logs](https://mlsanity.com/paper/2601.21043/) (2026)

## Abstract (arXiv)

Learning general-purpose user representations based on user behavioral logs is an increasingly popular user modeling approach. It benefits from easily available, privacy-friendly yet expressive data, and does not require extensive re-tuning of the upstream user model for different downstream tasks. While this approach has shown promise in search engines and e-commerce applications, its fit for instant messaging platforms, a cornerstone of modern digital communication, remains largely uncharted. We explore this research gap using Snapchat data as a case study. Specifically, we implement a Transformer-based user model with customized training objectives and show that the model can produce high-quality user representations across a broad range of evaluation tasks, among which we introduce three new downstream tasks that concern pivotal topics in user research: user safety, engagement and churn. We also tackle the challenge of efficient extrapolation of long sequences at inference time, by applying a novel positional encoding method.