# Exploring Bottom-Up Clustering for Creating Semantic IDs

arXiv: https://arxiv.org/abs/2609.08310 · HTML: https://mlsanity.com/paper/2609.08310/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 11 of its 13 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Builds on (most cited first, 11 of 11)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspective](https://mlsanity.com/paper/2404.12008/) (2024)
- [Generative Recommendation with Semantic IDs: A Practitioner's Handbook](https://mlsanity.com/paper/2507.22224/) (2025)
- [MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation](https://mlsanity.com/paper/2508.15281/) (2025)
- [Unified Semantic and ID Representation Learning for Deep Recommenders](https://mlsanity.com/paper/2502.16474/) (2025)
- [Breaking the Hourglass Phenomenon of Residual Quantization: Enhancing the Upper Bound of Generative Retrieval](https://mlsanity.com/paper/2407.21488/) (2024)
- [Understanding Generative Recommendation with Semantic IDs from a Model-scaling View](https://mlsanity.com/paper/2509.25522/) (2025)
- [CAT-ID2: Category-Tree Integrated Document Identifier Learning for Generative Retrieval In E-commerce](https://mlsanity.com/paper/2511.01461/) (2025)
- [Semantic IDs for Recommender Systems at Snapchat: Use Cases, Technical Challenges, and Design Choices](https://mlsanity.com/paper/2604.03949/) (2026)

## Abstract (arXiv)

The success of generative retrieval has largely been attributed to the use of Semantic IDs, which improve over arbitrary item-level identifiers such as hashes by capturing the semantics of items. The main challenges faced when constructing Semantic IDs, however, is in mapping each identifier to a unique product and capturing information valuable to downstream tasks. Past works have appended additional codewords to de-duplicate item identifiers and utilized residual quantization to create hierarchical clusters. In this work, we present an algorithm for generating Semantic IDs that ensure the identifiers are both unique and preserve the structure of the original embedding. Key to our work is the use of bottom-up clustering to preserve local structure in the embedding space, improving the clustering quality of the resulting Semantic IDs and their utility for downstream generative retrieval.