# MERGE: Next-Generation Item Indexing Paradigm for Large-Scale Streaming Recommendation

arXiv: https://arxiv.org/abs/2601.20199 · HTML: https://mlsanity.com/paper/2601.20199/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2026.

## Built on by (most cited first, 2 of 2)

- [Rethinking ANN-based Retrieval: Multifaceted Learnable Index for Large-scale Recommendation System](https://mlsanity.com/paper/2602.16124/) (2026)
- [From a Static Multi-Level Small Semantic Codebook to a Dynamic Single-Level Large Semantic Codebook for Generative Recommendation](https://mlsanity.com/paper/2608.21012/) (2026)

## Abstract (arXiv)

Item indexing, which maps a large corpus of items into compact discrete representations, is critical for both discriminative and generative recommender systems, yet existing Vector Quantization (VQ)-based approaches struggle with the highly skewed and non-stationary item distributions common in industrial streaming recommenders, leading to poor assignment accuracy, imbalanced cluster occupancy, and insufficient cluster separation. To address these challenges, we propose MERGE, a next-generation item indexing paradigm that adaptively constructs clusters from scratch, dynamically monitors cluster occupancy, and forms hierarchical index structures via fine-to-coarse merging. Extensive experiments demonstrate that MERGE significantly improves assignment accuracy, cluster uniformity, and cluster separation compared with existing indexing methods, while online A/B tests show substantial gains in key business metrics, highlighting its potential as a foundational indexing approach for large-scale recommendation. Codes are available at https://github.com/baiyimeng/MERGE.