# SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendation

arXiv: https://arxiv.org/abs/2608.18779 · HTML: https://mlsanity.com/paper/2608.18779/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 61 of its 72 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 30 of 61)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms](https://mlsanity.com/paper/2011.01731/) (2020)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [A Neural Corpus Indexer for Document Retrieval](https://mlsanity.com/paper/2206.02743/) (2022)
- [Learnable Item Tokenization for Generative Recommendation](https://mlsanity.com/paper/2405.07314/) (2024)
- [How to Index Item IDs for Recommendation Foundation Models](https://mlsanity.com/paper/2305.06569/) (2023)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research](https://mlsanity.com/paper/1911.07698/) (2019)
- [EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration](https://mlsanity.com/paper/2406.14017/) (2024)
- [Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations](https://mlsanity.com/paper/2306.08121/) (2023)
- [TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendation](https://mlsanity.com/paper/2406.10450/) (2024)
- [Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations](https://mlsanity.com/paper/2503.02453/) (2025)
- [Generative Recommender with End-to-End Learnable Item Tokenization](https://mlsanity.com/paper/2409.05546/) (2024)
- [Elliot: a Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluation](https://mlsanity.com/paper/2103.02590/) (2021)
- [QARM: Quantitative Alignment Multi-Modal Recommendation at Kuaishou](https://mlsanity.com/paper/2411.11739/) (2024)
- [CoST: Contrastive Quantization based Semantic Tokenization for Generative Recommendation](https://mlsanity.com/paper/2404.14774/) (2024)
- [Differentiable Semantic ID for Generative Recommendation](https://mlsanity.com/paper/2601.19711/) (2026)
- [Generative Recommendation with Semantic IDs: A Practitioner's Handbook](https://mlsanity.com/paper/2507.22224/) (2025)
- [Semantic IDs for Joint Generative Search and Recommendation](https://mlsanity.com/paper/2508.10478/) (2025)
- [LensKit for Python: Next-Generation Software for Recommender System Experiments](https://mlsanity.com/paper/1809.03125/) (2018)
- [Multimodal Quantitative Language for Generative Recommendation](https://mlsanity.com/paper/2504.05314/) (2025)
- [From IDs to Semantics: A Generative Framework for Cross-Domain Recommendation with Adaptive Semantic Tokenization](https://mlsanity.com/paper/2511.08006/) (2025)
- [Beyond NDCG: behavioral testing of recommender systems with RecList](https://mlsanity.com/paper/2111.09963/) (2021)
- [DiscRec: Disentangled Semantic-Collaborative Modeling for Generative Recommendation](https://mlsanity.com/paper/2506.15576/) (2025)

## Abstract (arXiv)

Semantic ID tokenizers assign code sequences to catalog items, which generative recommenders predict and resolve back to items. Reusing a released mapping requires checking its code organization, item resolution, and compatibility with an existing model after a refresh. SIDScope provides these checks through source traced adapters, mapping diagnostics, candidate retrieval tests, and generated trace analysis. Its inventory contains nine exports from seven tokenizer families across Amazon and Yelp, including eight executable routes and one auditable snapshot. The analyses establish three distinctions. First, code allocation and item addressability differ: in GRID, tail items occupy 95.7 distinct codes per 100 items, yet only 82.2 percent have globally unique codes. Second, prefix alignment calibrates retrieval operations that use prefixes. With fixed candidate budgets and ranking, original assignments on Beauty and Instruments expose more targets than controls preserving code capacity and coarse popularity. Resolving identical generated paths also shows that an explicit item selection rule recovers part of the gap between target path survival and singleton credit. Third, a released refresh repairs missing codes, while model adaptation separately restores new item recall. SIDScope supplies code, diagnostic summaries, and verification tools to compare mappings, evaluate their use in retrieval and generation, and test model reuse after changes.