# TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendation

arXiv: https://arxiv.org/abs/2406.10450 · HTML: https://mlsanity.com/paper/2406.10450/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 77 references that are in the map.
- First posted on arXiv 15 Jun 2024.

## Builds on (most cited first, 30 of 46)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://mlsanity.com/paper/2307.03393/) (2023)
- [Can Language Models Solve Graph Problems in Natural Language?](https://mlsanity.com/paper/2305.10037/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis](https://mlsanity.com/paper/2211.09117/) (2022)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [How to Index Item IDs for Recommendation Foundation Models](https://mlsanity.com/paper/2305.06569/) (2023)
- [CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation](https://mlsanity.com/paper/2310.19488/) (2023)

## Built on by (most cited first, 30 of 35)

- [Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation](https://mlsanity.com/paper/2501.02226/) (2025)
- [A Survey of Mamba](https://mlsanity.com/paper/2408.01129/) (2024)
- [MiniOneRec: An Open-Source Framework for Scaling Generative Recommendation](https://mlsanity.com/paper/2510.24431/) (2025)
- [MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation](https://mlsanity.com/paper/2508.15281/) (2025)
- [Enhancing Item Tokenization for Generative Recommendation through Self-Improvement](https://mlsanity.com/paper/2412.17171/) (2024)
- [Unleash LLMs Potential for Sequential Recommendation by Coordinating Dual Dynamic Index Mechanism](https://mlsanity.com/paper/2409.09253/) (2024)
- [Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey](https://mlsanity.com/paper/2507.22920/) (2025)
- [Retrieval-Augmented Purifier for Robust LLM-Empowered Recommendation](https://mlsanity.com/paper/2504.02458/) (2025)
- [Universal Item Tokenization for Transferable Generative Recommendation](https://mlsanity.com/paper/2504.04405/) (2025)
- [Masked Diffusion for Generative Recommendation](https://mlsanity.com/paper/2511.23021/) (2025)
- [Generative Recommendation for Large-Scale Advertising](https://mlsanity.com/paper/2602.22732/) (2026)
- [Computational Protein Science in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2501.10282/) (2025)
- [Exploring Backdoor Attack and Defense for LLM-empowered Recommendations](https://mlsanity.com/paper/2504.11182/) (2025)
- [Chain-of-Query: Unleashing the Power of LLMs in SQL-Aided Table Understanding via Multi-Agent Collaboration](https://mlsanity.com/paper/2508.15809/) (2025)
- [Pre-training Generative Recommender with Multi-Identifier Item Tokenization](https://mlsanity.com/paper/2504.04400/) (2025)
- [A Simple Contrastive Framework Of Item Tokenization For Generative Recommendation](https://mlsanity.com/paper/2506.16683/) (2025)
- [Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation](https://mlsanity.com/paper/2503.05493/) (2025)
- [Hippocampus: An Efficient and Scalable Memory Module for Agentic AI](https://mlsanity.com/paper/2602.13594/) (2026)
- [Enhancing Molecular Property Predictions by Learning from Bond Modelling and Interactions](https://mlsanity.com/paper/2603.00568/) (2026)
- [An Embarrassingly Simple Graph Heuristic Reveals Shortcut-Solvable Benchmarks for Sequential Recommendation](https://mlsanity.com/paper/2605.07125/) (2026)
- [SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation](https://mlsanity.com/paper/2605.18920/) (2026)
- [ChronoID: Infusing Explicit Temporal Signals into Semantic IDs for Generative Recommendation](https://mlsanity.com/paper/2606.14260/) (2026)
- [Token Factory: Efficiently Integrating Diverse Signals into Large Recommendation Models](https://mlsanity.com/paper/2606.19635/) (2026)
- [DeMa: Dual-Path Delay-Aware Mamba for Efficient Multivariate Time Series Analysis](https://mlsanity.com/paper/2601.05527/) (2026)
- [PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendation](https://mlsanity.com/paper/2601.16556/) (2026)
- [One Model, Two Markets: Bid-Aware Generative Recommendation](https://mlsanity.com/paper/2603.22231/) (2026)
- [PriHA: A RAG-Enhanced LLM Framework for Primary Healthcare Assistant in Hong Kong](https://mlsanity.com/paper/2604.14215/) (2026)
- [Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation](https://mlsanity.com/paper/2604.27747/) (2026)
- [Diffusion Language Model for Recommendation](https://mlsanity.com/paper/2607.21519/) (2026)
- [Attribute-Conditioned Multimodal Slot Factorization for Controllable Fashion Retrieval](https://mlsanity.com/paper/2608.12570/) (2026)

## Abstract (arXiv)

There is a growing interest in utilizing large-scale language models (LLMs) to advance next-generation Recommender Systems (RecSys), driven by their outstanding language understanding and in-context learning capabilities. In this scenario, tokenizing (i.e., indexing) users and items becomes essential for ensuring a seamless alignment of LLMs with recommendations. While several studies have made progress in representing users and items through textual contents or latent representations, challenges remain in efficiently capturing high-order collaborative knowledge into discrete tokens that are compatible with LLMs. Additionally, the majority of existing tokenization approaches often face difficulties in generalizing effectively to new/unseen users or items that were not in the training corpus. To address these challenges, we propose a novel framework called TokenRec, which introduces not only an effective ID tokenization strategy but also an efficient retrieval paradigm for LLM-based recommendations. Specifically, our tokenization strategy, Masked Vector-Quantized (MQ) Tokenizer, involves quantizing the masked user/item representations learned from collaborative filtering into discrete tokens, thus achieving a smooth incorporation of high-order collaborative knowledge and a generalizable tokenization of users and items for LLM-based RecSys. Meanwhile, our generative retrieval paradigm is designed to efficiently recommend top-K items for users to eliminate the need for the time-consuming auto-regressive decoding and beam search processes used by LLMs, thus significantly reducing inference time. Comprehensive experiments validate the effectiveness of the proposed methods, demonstrating that TokenRec outperforms competitive benchmarks, including both traditional recommender systems and emerging LLM-based recommender systems.