# Behavior Sequence Transformer for E-commerce Recommendation in Alibaba

arXiv: https://arxiv.org/abs/1905.06874 · HTML: https://mlsanity.com/paper/1905.06874/

## Key facts

- Cited by 130 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2019.

## Built on by (most cited first, 30 of 130)

- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [A Review of Sparse Expert Models in Deep Learning](https://mlsanity.com/paper/2209.01667/) (2022)
- [A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity](https://mlsanity.com/paper/2302.06015/) (2023)
- [Max-Margin Token Selection in Attention Mechanism](https://mlsanity.com/paper/2306.13596/) (2023)
- [PinnerFormer: Sequence Modeling for User Representation at Pinterest](https://mlsanity.com/paper/2205.04507/) (2022)
- [ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models](https://mlsanity.com/paper/2305.06566/) (2023)
- [Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation](https://mlsanity.com/paper/2503.22675/) (2025)
- [TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest](https://mlsanity.com/paper/2306.00248/) (2023)
- [Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations](https://mlsanity.com/paper/2503.02453/) (2025)
- [Deep Learning for Click-Through Rate Estimation](https://mlsanity.com/paper/2104.10584/) (2021)
- [ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs](https://mlsanity.com/paper/2210.03052/) (2022)
- [Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem](https://mlsanity.com/paper/2110.02629/) (2021)
- [EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate](https://mlsanity.com/paper/2112.14397/) (2021)
- [TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou](https://mlsanity.com/paper/2407.16357/) (2024)
- [PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations](https://mlsanity.com/paper/2510.07784/) (2025)
- [TabGNN: Multiplex Graph Neural Network for Tabular Data Prediction](https://mlsanity.com/paper/2108.09127/) (2021)
- [Scaling Law for Recommendation Models: Towards General-purpose User Representations](https://mlsanity.com/paper/2111.11294/) (2021)
- [A Survey on User Behavior Modeling in Recommender Systems](https://mlsanity.com/paper/2302.11087/) (2023)
- [Rethinking Personalized Ranking at Pinterest: An End-to-End Approach](https://mlsanity.com/paper/2209.08435/) (2022)
- [MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems](https://mlsanity.com/paper/2310.02784/) (2023)
- [FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data](https://mlsanity.com/paper/2310.13818/) (2023)
- [What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding](https://mlsanity.com/paper/2406.01977/) (2024)
- [Ads Recommendation in a Collapsed and Entangled World](https://mlsanity.com/paper/2403.00793/) (2024)
- [PEAR: Personalized Re-ranking with Contextualized Transformer for Recommendation](https://mlsanity.com/paper/2203.12267/) (2022)
- [LiRank: Industrial Large Scale Ranking Models at LinkedIn](https://mlsanity.com/paper/2402.06859/) (2024)
- [KEEP: An Industrial Pre-Training Framework for Online Recommendation via Knowledge Extraction and Plugging](https://mlsanity.com/paper/2208.10174/) (2022)
- [Causal Decision Transformer for Recommender Systems via Offline Reinforcement Learning](https://mlsanity.com/paper/2304.07920/) (2023)
- [Out of the Box Thinking: Improving Customer Lifetime Value Modelling via Expert Routing and Game Whale Detection](https://mlsanity.com/paper/2308.12729/) (2023)
- [Hierarchical LoRA MoE for Efficient CTR Model Scaling](https://mlsanity.com/paper/2510.10432/) (2025)
- [RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure](https://mlsanity.com/paper/2211.05239/) (2022)

## Abstract (arXiv)

Deep learning based methods have been widely used in industrial recommendation systems (RSs). Previous works adopt an Embedding&MLP paradigm: raw features are embedded into low-dimensional vectors, which are then fed on to MLP for final recommendations. However, most of these works just concatenate different features, ignoring the sequential nature of users' behaviors. In this paper, we propose to use the powerful Transformer model to capture the sequential signals underlying users' behavior sequences for recommendation in Alibaba. Experimental results demonstrate the superiority of the proposed model, which is then deployed online at Taobao and obtain significant improvements in online Click-Through-Rate (CTR) comparing to two baselines.