# Variable-Length Semantic IDs for Recommender Systems

arXiv: https://arxiv.org/abs/2602.16375 · HTML: https://mlsanity.com/paper/2602.16375/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 51 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Primer: Searching for Efficient Transformers for Language Modeling](https://mlsanity.com/paper/2109.08668/) (2021)
- [Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols](https://mlsanity.com/paper/1705.11192/) (2017)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Anti-efficient encoding in emergent communication](https://mlsanity.com/paper/1905.12561/) (2019)
- [Extending Context Window of Large Language Models via Semantic Compression](https://mlsanity.com/paper/2312.09571/) (2023)
- [EGG: a toolkit for research on Emergence of lanGuage in Games](https://mlsanity.com/paper/1907.00852/) (2019)
- [PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations](https://mlsanity.com/paper/2510.07784/) (2025)
- [Semantic IDs for Joint Generative Search and Recommendation](https://mlsanity.com/paper/2508.10478/) (2025)
- [Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders](https://mlsanity.com/paper/2507.16289/) (2025)
- [Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments](https://mlsanity.com/paper/2311.04453/) (2023)
- [Scaling Recommender Transformers to One Billion Parameters](https://mlsanity.com/paper/2507.15994/) (2025)
- [Yambda-5B -- A Large-Scale Multi-modal Dataset for Ranking And Retrieval](https://mlsanity.com/paper/2505.22238/) (2025)
- [VK-LSVD: A Large-Scale Industrial Dataset for Short-Video Recommendation](https://mlsanity.com/paper/2602.04567/) (2026)

## Built on by (most cited first, 4 of 4)

- [HoloRec: Holistic Encoding and Interleaved Reasoning for Generative Recommendation](https://mlsanity.com/paper/2606.15331/) (2026)
- [Learning from Unreachable Rewards: Hint-Conditioned Reinforcement Learning for Generative Recommendation](https://mlsanity.com/paper/2608.11980/) (2026)
- [SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendation](https://mlsanity.com/paper/2608.18779/) (2026)
- [Neither Black nor White: Balancing Semantic and Collaborative Signals with Graph-Informed Semantic IDs (GrIS)](https://mlsanity.com/paper/2610.01533/) (2026)

## Abstract (arXiv)

Generative models are increasingly used in recommender systems, both for modeling user behavior as event sequences and for integrating large language models into recommendation pipelines. A key challenge in this setting is the extremely large cardinality of item spaces, which makes training generative models difficult and introduces a vocabulary gap between natural language and item identifiers. Semantic identifiers (semantic IDs), which represent items as sequences of low-cardinality tokens, have recently emerged as an effective solution to this problem. However, existing approaches generate semantic identifiers of fixed length, assigning the same description length to all items. This is inefficient, misaligned with natural language, and ignores the highly skewed frequency structure of real-world catalogs, where popular items and rare long-tail items exhibit fundamentally different information requirements. In parallel, the emergent communication literature studies how agents develop discrete communication protocols, often producing variable-length messages in which frequent concepts receive shorter descriptions. Despite the conceptual similarity, these ideas have not been systematically adopted in recommender systems. In this work, we bridge recommender systems and emergent communication by introducing variable-length semantic identifiers for recommendation. We propose a discrete variational autoencoder with Gumbel-Softmax reparameterization that learns item representations of adaptive length under a principled probabilistic framework, avoiding the instability of REINFORCE-based training and the fixed-length constraints of prior semantic ID methods.