# TRACE: Transformer-based user Representations from Attributed Clickstream Event sequences

arXiv: https://arxiv.org/abs/2409.12972 · HTML: https://mlsanity.com/paper/2409.12972/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [PinnerFormer: Sequence Modeling for User Representation at Pinterest](https://mlsanity.com/paper/2205.04507/) (2022)
- [Large Language Models for User Interest Journeys](https://mlsanity.com/paper/2305.15498/) (2023)
- [Optimizing Airbnb Search Journey with Multi-task Learning](https://mlsanity.com/paper/2305.18431/) (2023)
- [Recommending Complementary Products in E-Commerce Push Notifications with a Mixture Model Approach](https://mlsanity.com/paper/1707.08113/) (2017)
- [Attributed Sequence Embedding](https://mlsanity.com/paper/1911.00949/) (2019)
- [Learning to Personalize Recommendation based on Customers' Shopping Intents](https://mlsanity.com/paper/2305.05279/) (2023)
- [AdaptEx: A Self-Service Contextual Bandit Platform](https://mlsanity.com/paper/2308.08650/) (2023)

## Built on by (most cited first, 3 of 3)

- [SimPersona: Learning Discrete Buyer Personas from Raw Clickstreams for Grounded E-Commerce Agents](https://mlsanity.com/paper/2605.14205/) (2026)
- [From Clicks to Intent: Cross-Platform Session Embeddings with LLM-Distilled Taxonomy for Financial Services Recommendations](https://mlsanity.com/paper/2606.26277/) (2026)
- [Behavioral Fingerprinting and Navigation Prediction in Web Browsing](https://mlsanity.com/paper/2609.18273/) (2026)

## Abstract (arXiv)

For users navigating travel e-commerce websites, the process of researching products and making a purchase often results in intricate browsing patterns that span numerous sessions over an extended period of time. The resulting clickstream data chronicle these user journeys and present valuable opportunities to derive insights that can significantly enhance personalized recommendations. We introduce TRACE, a novel transformer-based approach tailored to generate rich user embeddings from live multi-session clickstreams for real-time recommendation applications. Prior works largely focus on single-session product sequences, whereas TRACE leverages site-wide page view sequences spanning multiple user sessions to model long-term engagement. Employing a multi-task learning framework, TRACE captures comprehensive user preferences and intents distilled into low-dimensional representations. We demonstrate TRACE's superior performance over vanilla transformer and LLM-style architectures through extensive experiments on a large-scale travel e-commerce dataset of real user journeys, where the challenges of long page-histories and sparse targets are particularly prevalent. Visualizations of the learned embeddings reveal meaningful clusters corresponding to latent user states and behaviors, highlighting TRACE's potential to enhance recommendation systems by capturing nuanced user interactions and preferences