# Generative Recommender with End-to-End Learnable Item Tokenization

arXiv: https://arxiv.org/abs/2409.05546 · HTML: https://mlsanity.com/paper/2409.05546/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2024.

## Built on by (most cited first, 30 of 34)

- [MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation](https://mlsanity.com/paper/2508.15281/) (2025)
- [LLaDA-Rec: Discrete Diffusion for Parallel Semantic ID Generation in Generative Recommendation](https://mlsanity.com/paper/2511.06254/) (2025)
- [Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey](https://mlsanity.com/paper/2507.22920/) (2025)
- [Universal Item Tokenization for Transferable Generative Recommendation](https://mlsanity.com/paper/2504.04405/) (2025)
- [Understanding Generative Recommendation with Semantic IDs from a Model-scaling View](https://mlsanity.com/paper/2509.25522/) (2025)
- [Stop Treating Collisions Equally: Qualification-Aware Semantic ID Learning for Recommendation at Industrial Scale](https://mlsanity.com/paper/2603.00632/) (2026)
- [Pctx: Tokenizing Personalized Context for Generative Recommendation](https://mlsanity.com/paper/2510.21276/) (2025)
- [When Text-as-Vision Meets Semantic IDs in Generative Recommendation: An Empirical Study](https://mlsanity.com/paper/2601.14697/) (2026)
- [Generative Recall, Dense Reranking: Learning Multi-View Semantic IDs for Efficient Text-to-Video Retrieval](https://mlsanity.com/paper/2601.21193/) (2026)
- [Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation](https://mlsanity.com/paper/2607.24995/) (2026)
- [Cognitive-Aligned Spatio-Temporal Large Language Models For Next Point-of-Interest Prediction](https://mlsanity.com/paper/2510.14702/) (2025)
- [Rethinking Generative Recommender Tokenizer: Recsys-Native Encoding and Semantic Quantization Beyond LLMs](https://mlsanity.com/paper/2602.02338/) (2026)
- [MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches](https://mlsanity.com/paper/2604.22881/) (2026)
- [LLMs Need Encoders for Semantic IDs Too](https://mlsanity.com/paper/2606.00324/) (2026)
- [TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation](https://mlsanity.com/paper/2606.07688/) (2026)
- [HoMer: Addressing Heterogeneities by Modeling Sequential and Set-wise Contexts for CTR Prediction](https://mlsanity.com/paper/2510.11100/) (2025)
- [UniGRec: Unified Generative Recommendation with Soft Identifiers for End-to-End Optimization](https://mlsanity.com/paper/2601.17438/) (2026)
- [Learning Variable-Length Tokenization for Generative Recommendation](https://mlsanity.com/paper/2605.17779/) (2026)
- [Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulness](https://mlsanity.com/paper/2608.10008/) (2026)
- [PRQ-KMeans: Projection Residual Quantization for Semantic ID Tokenization](https://mlsanity.com/paper/2608.24207/) (2026)
- [History-Conditioned Joint-Prefix Alignment for Generative Recommendation](https://mlsanity.com/paper/2608.29179/) (2026)
- [SPARC: Soft Probabilistic Adaptive multi-interest Retrieval Model via Codebooks for recommender system](https://mlsanity.com/paper/2508.09090/) (2025)
- [Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation](https://mlsanity.com/paper/2509.10468/) (2025)
- [PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendation](https://mlsanity.com/paper/2601.16556/) (2026)
- [SIGMA: A Semantic-Grounded Instruction-Driven Generative Multi-Task Recommender at AliExpress](https://mlsanity.com/paper/2602.22913/) (2026)
- [VaLiDRec: Variable-Length LLM-Aligned Semantic IDs for Generative Recommendation](https://mlsanity.com/paper/2607.25209/) (2026)
- [EchoRec: Multi-Item Prediction-Empowered Generative Recommendation via Cycle-Consistent Preference Alignment](https://mlsanity.com/paper/2608.14011/) (2026)
- [SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendation](https://mlsanity.com/paper/2608.18779/) (2026)
- [From a Static Multi-Level Small Semantic Codebook to a Dynamic Single-Level Large Semantic Codebook for Generative Recommendation](https://mlsanity.com/paper/2608.21012/) (2026)
- [SA-RSQ: A Versatile Sparse Representation Framework for Multi-modal Recommender Systems](https://mlsanity.com/paper/2608.22979/) (2026)

## Abstract (arXiv)

Generative recommendation systems have gained increasing attention as an innovative approach that directly generates item identifiers for recommendation tasks. Despite their potential, a major challenge is the effective construction of item identifiers that align well with recommender systems. Current approaches often treat item tokenization and generative recommendation training as separate processes, which can lead to suboptimal performance. To overcome this issue, we introduce ETEGRec, a novel End-To-End Generative Recommender that unifies item tokenization and generative recommendation into a cohesive framework. Built on a dual encoder-decoder architecture, ETEGRec consists of an item tokenizer and a generative recommender. To enable synergistic interaction between these components, we propose a recommendation-oriented alignment strategy, which includes two key optimization objectives: sequence-item alignment and preference-semantic alignment. These objectives tightly couple the learning processes of the item tokenizer and the generative recommender, fostering mutual enhancement. Additionally, we develop an alternating optimization technique to ensure stable and efficient end-to-end training of the entire framework. Extensive experiments demonstrate the superior performance of our approach compared to traditional sequential recommendation models and existing generative recommendation baselines. Our code is available at https://github.com/RUCAIBox/ETEGRec.