# ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation

arXiv: https://arxiv.org/abs/2502.13581 · HTML: https://mlsanity.com/paper/2502.13581/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 77 references that are in the map.
- First posted on arXiv 19 Feb 2025.

## Builds on (most cited first, 30 of 62)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [BPE-Dropout: Simple and Effective Subword Regularization](https://mlsanity.com/paper/1910.13267/) (2019)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)

## Built on by (most cited first, 18 of 18)

- [RecoWorld: Building Simulated Environments for Agentic Recommender Systems](https://mlsanity.com/paper/2509.10397/) (2025)
- [CoFiRec: Coarse-to-Fine Tokenization for Generative Recommendation](https://mlsanity.com/paper/2511.22707/) (2025)
- [MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation](https://mlsanity.com/paper/2508.15281/) (2025)
- [Pre-training Generative Recommender with Multi-Identifier Item Tokenization](https://mlsanity.com/paper/2504.04400/) (2025)
- [Act-With-Think: Chunk Auto-Regressive Modeling for Generative Recommendation](https://mlsanity.com/paper/2506.23643/) (2025)
- [GRACE: Generative Recommendation via Journey-Aware Sparse Attention on Chain-of-Thought Tokenization](https://mlsanity.com/paper/2507.14758/) (2025)
- [DiffGRM: Diffusion-based Generative Recommendation Model](https://mlsanity.com/paper/2510.21805/) (2025)
- [Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation](https://mlsanity.com/paper/2509.26063/) (2025)
- [Pctx: Tokenizing Personalized Context for Generative Recommendation](https://mlsanity.com/paper/2510.21276/) (2025)
- [Generative Recall, Dense Reranking: Learning Multi-View Semantic IDs for Efficient Text-to-Video Retrieval](https://mlsanity.com/paper/2601.21193/) (2026)
- [An Embarrassingly Simple Graph Heuristic Reveals Shortcut-Solvable Benchmarks for Sequential Recommendation](https://mlsanity.com/paper/2605.07125/) (2026)
- [ChronoID: Infusing Explicit Temporal Signals into Semantic IDs for Generative Recommendation](https://mlsanity.com/paper/2606.14260/) (2026)
- [T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation](https://mlsanity.com/paper/2609.30576/) (2026)
- [Token-Efficient Item Representation via Images for LLM Recommender Systems](https://mlsanity.com/paper/2503.06238/) (2025)
- [Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation](https://mlsanity.com/paper/2604.27747/) (2026)
- [On the Memorization Behavior of LLMs in Generative Recommendation: Observations, Implications, and Training Strategies](https://mlsanity.com/paper/2606.17276/) (2026)
- [SAGA: Structure-Attended Generative Action Embedding Model that encodes Multi-Surface User Action Sequences](https://mlsanity.com/paper/2608.15429/) (2026)
- [Towards a Densing Law for User Representation Learning at Billion-Scale Capacity](https://mlsanity.com/paper/2608.23392/) (2026)

## Abstract (arXiv)

Generative recommendation (GR) is an emerging paradigm where user actions are tokenized into discrete token patterns and autoregressively generated as predictions. However, existing GR models tokenize each action independently, assigning the same fixed tokens to identical actions across all sequences without considering contextual relationships. This lack of context-awareness can lead to suboptimal performance, as the same action may hold different meanings depending on its surrounding context. To address this issue, we propose ActionPiece to explicitly incorporate context when tokenizing action sequences. In ActionPiece, each action is represented as a set of item features. Given the action sequence corpora, we construct the vocabulary by merging feature patterns as new tokens, based on their co-occurrence frequency both within individual sets and across adjacent sets. Considering the unordered nature of feature sets, we further introduce set permutation regularization, which produces multiple segmentations of action sequences with the same semantics. Our code is available at: https://github.com/google-deepmind/action_piece.