Stop Treating Collisions Equally: Qualification-Aware Semantic ID Learning for Recommendation at Industrial Scale
Not on the map. Why?
Abstract (arXiv)
Semantic IDs (SIDs) are compact discrete representations derived from multimodal item features, serving as a unified abstraction for ID-based and generative recommendation. However, learning high-quality SIDs remains challenging due to two issues. (1) Collision problem: the quantized token space is prone to collisions, in which semantically distinct items are assigned identical or overly similar SID compositions, resulting in semantic entanglement. (2) Collision-signal heterogeneity: collisions are not uniformly harmful. Some reflect genuine conflicts between semantically unrelated items, while others stem from benign redundancy or systematic data effects. To address these challenges, we propose Qualification-Aware Semantic ID Learning (QuaSID), an end-to-end framework that learns collision-qualified SIDs by selectively repelling qualified conflict pairs and scaling the repulsion strength by collision severity. QuaSID consists of two mechanisms: Hamming-guided Margin Repulsion, which translates low-Hamming SID overlaps into explicit, severity-scaled geometric constraints on the encoder space; and Conflict-Aware Valid Pair Masking, which masks protocol-induced benign overlaps to denoise repulsion supervision. In addition, QuaSID incorporates a dual-tower contrastive objective to inject collaborative signals into tokenization. Experiments on public benchmarks and industrial data validate QuaSID. On public datasets, QuaSID consistently outperforms strong baselines, improving top-K ranking quality by 5.9% over the best baseline while increasing SID composition diversity. In an online A/B test on Kuaishou e-commerce with a 5% traffic split, QuaSID increases ranking GMV-S2 by 2.38% and improves completed orders on cold-start retrieval by up to 6.42%. Finally, we show that the proposed repulsion loss is plug-and-play and enhances a range of SID learning frameworks across datasets.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 27 of its 34 references that are in the map. First posted on arXiv 28 Feb 2026.
Builds on
27 of 34 references.
References · 27
- 2017Neural Discrete Representation Learningcited by 4,650
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2022Autoregressive Image Generation using Residual Quantizationcited by 706
- 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
- 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
- 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
- 2023Adapting Large Language Models by Integrating Collaborative Semantics for Recommendationcited by 171
- 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 158
- 2022Learning Vector-Quantized Item Representation for Transferable Sequential Recommenderscited by 112
- 2024Learnable Item Tokenization for Generative Recommendationcited by 106
15 more
- 2023HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codeccited by 95
- 2023Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendationscited by 50
- 2024Restructuring Vector Quantization with the Rotation Trickcited by 49
- 2025MTGR: Industrial-Scale Generative Recommendation Framework in Meituancited by 47
- 2024Generative Recommender with End-to-End Learnable Item Tokenizationcited by 32
- 2025Generating Long Semantic IDs in Parallel for Recommendationcited by 29
- 2020Large Scale Product Graph Construction for Recommendation in E-commercecited by 28
- 2025OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Searchcited by 24
- 2025Enhancing Embedding Representation Stability in Recommendation Systems with Semantic IDcited by 19
- 2021NWT: Towards natural audio-to-video generation with representation learningcited by 14
- 2025From Principles to Applications: A Comprehensive Survey of Discrete Tokenizers in Generation, Comprehension, Recommendation, and Information Retrievalcited by 11
- 2025HiD-VAE: Interpretable Generative Recommendation via Hierarchical and Disentangled Semantic IDscited by 11
- 2025Real-time Indexing for Large-scale Recommendation by Streaming Vector Quantization Retrievercited by 9
- 2025DAS: Dual-Aligned Semantic IDs Empowered Industrial Recommender Systemcited by 8
- 2025Bridging Textual-Collaborative Gap through Semantic Codes for Sequential Recommendationcited by 4
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain Recommendationscited by 2
- 2026A/B Agent: A Self-Evolving Agent for Strategy Iteration in Industrial A/B Testingcited by 1
- 2026Conditional Memory Enhanced Item Representation for Generative Recommendationcited by 0
- 2026SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendationcited by 0
- 2026Guiding the coarse levels of semantic IDs makes the fine levels learnablecited by 0
- 2026Guiding the coarse levels of semantic IDs makes the fine levels learnable3 Sep 2026
- 2026SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendation19 Aug 2026
- 2026A/B Agent: A Self-Evolving Agent for Strategy Iteration in Industrial A/B Testing5 Aug 2026
- 2026Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain Recommendations9 Jun 2026
- 2026Conditional Memory Enhanced Item Representation for Generative Recommendation12 May 2026