# CoST: Contrastive Quantization based Semantic Tokenization for Generative Recommendation

arXiv: https://arxiv.org/abs/2404.14774 · HTML: https://mlsanity.com/paper/2404.14774/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2024.

## Built on by (most cited first, 26 of 26)

- [Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations](https://mlsanity.com/paper/2503.02453/) (2025)
- [ActionPiece: Contextually Tokenizing Action Sequences for Generative Recommendation](https://mlsanity.com/paper/2502.13581/) (2025)
- [Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey](https://mlsanity.com/paper/2507.22920/) (2025)
- [Masked Diffusion for Generative Recommendation](https://mlsanity.com/paper/2511.23021/) (2025)
- [Generative Recommendation for Large-Scale Advertising](https://mlsanity.com/paper/2602.22732/) (2026)
- [NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations](https://mlsanity.com/paper/2511.18793/) (2025)
- [DeepInterestGR: Mining Deep Multi-Interest Using Multi-Modal LLMs for Generative Recommendation](https://mlsanity.com/paper/2602.18907/) (2026)
- [Large Language Model as Universal Retriever in Industrial-Scale Recommender System](https://mlsanity.com/paper/2502.03041/) (2025)
- [A Simple Contrastive Framework Of Item Tokenization For Generative Recommendation](https://mlsanity.com/paper/2506.16683/) (2025)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [Pctx: Tokenizing Personalized Context for Generative Recommendation](https://mlsanity.com/paper/2510.21276/) (2025)
- [MVIGER: Multi-View Variational Integration of Complementary Knowledge for Generative Recommender](https://mlsanity.com/paper/2408.08686/) (2024)
- [RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation](https://mlsanity.com/paper/2509.03131/) (2025)
- [Rethinking Generative Recommender Tokenizer: Recsys-Native Encoding and Semantic Quantization Beyond LLMs](https://mlsanity.com/paper/2602.02338/) (2026)
- [A Unified Language Model for Large Scale Search, Recommendation, and Reasoning](https://mlsanity.com/paper/2603.17533/) (2026)
- [LLMs Need Encoders for Semantic IDs Too](https://mlsanity.com/paper/2606.00324/) (2026)
- [Can Generative Recommendation Reach Cold Items? A Temporal Perspective on Semantic-ID Generation](https://mlsanity.com/paper/2607.21101/) (2026)
- [GenRecEdit: Adapting Model Editing for Generative Recommendation with Cold-Start Items](https://mlsanity.com/paper/2603.14259/) (2026)
- [Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation](https://mlsanity.com/paper/2608.11980/) (2026)
- [PRQ-KMeans: Projection Residual Quantization for Semantic ID Tokenization](https://mlsanity.com/paper/2608.24207/) (2026)
- [Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation](https://mlsanity.com/paper/2509.10468/) (2025)
- [Trie-Aware Transformers for Generative Recommendation](https://mlsanity.com/paper/2602.21677/) (2026)
- [VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models](https://mlsanity.com/paper/2603.12625/) (2026)
- [Conditional Memory Enhanced Item Representation for Generative Recommendation](https://mlsanity.com/paper/2605.11447/) (2026)
- [SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendation](https://mlsanity.com/paper/2608.18779/) (2026)
- [Towards a Densing Law for User Representation Learning at Billion-Scale Capacity](https://mlsanity.com/paper/2608.23392/) (2026)

## Abstract (arXiv)

Embedding-based retrieval serves as a dominant approach to candidate item matching for industrial recommender systems. With the success of generative AI, generative retrieval has recently emerged as a new retrieval paradigm for recommendation, which casts item retrieval as a generation problem. Its model consists of two stages: semantic tokenization and autoregressive generation. The first stage involves item tokenization that constructs discrete semantic tokens to index items, while the second stage autoregressively generates semantic tokens of candidate items. Therefore, semantic tokenization serves as a crucial preliminary step for training generative recommendation models. Existing research usually employs a vector quantizier with reconstruction loss (e.g., RQ-VAE) to obtain semantic tokens of items, but this method fails to capture the essential neighborhood relationships that are vital for effective item modeling in recommender systems. In this paper, we propose a contrastive quantization-based semantic tokenization approach, named CoST, which harnesses both item relationships and semantic information to learn semantic tokens. Our experimental results highlight the significant impact of semantic tokenization on generative recommendation performance, with CoST achieving up to a 43% improvement in Recall@5 and 44% improvement in NDCG@5 on the MIND dataset over previous baselines.