# Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations

arXiv: https://arxiv.org/abs/2503.02453 · HTML: https://mlsanity.com/paper/2503.02453/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 57 references that are in the map.
- First posted on arXiv 4 Mar 2025.

## Builds on (most cited first, 30 of 53)

- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [Recurrent Neural Network Regularization](https://mlsanity.com/paper/1409.2329/) (2014)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [Text Is All You Need: Learning Language Representations for Sequential Recommendation](https://mlsanity.com/paper/2305.13731/) (2023)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [Towards Universal Sequence Representation Learning for Recommender Systems](https://mlsanity.com/paper/2206.05941/) (2022)
- [Learnable Item Tokenization for Generative Recommendation](https://mlsanity.com/paper/2405.07314/) (2024)
- [How to Index Item IDs for Recommendation Foundation Models](https://mlsanity.com/paper/2305.06569/) (2023)
- [Hierarchical Gating Networks for Sequential Recommendation](https://mlsanity.com/paper/1906.09217/) (2019)
- [Joint Deep Modeling of Users and Items Using Reviews for Recommendation](https://mlsanity.com/paper/1701.04783/) (2017)
- [A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems](https://mlsanity.com/paper/2308.08434/) (2023)
- [GenRec: Large Language Model for Generative Recommendation](https://mlsanity.com/paper/2307.00457/) (2023)
- [HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling](https://mlsanity.com/paper/2409.12740/) (2024)
- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration](https://mlsanity.com/paper/2406.14017/) (2024)
- [PinnerFormer: Sequence Modeling for User Representation at Pinterest](https://mlsanity.com/paper/2205.04507/) (2022)

## Built on by (most cited first, 30 of 35)

- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [MMQ: Multimodal Mixture-of-Quantization Tokenization for Semantic ID Generation and User Behavioral Adaptation](https://mlsanity.com/paper/2508.15281/) (2025)
- [Masked Diffusion for Generative Recommendation](https://mlsanity.com/paper/2511.23021/) (2025)
- [Understanding Generative Recommendation with Semantic IDs from a Model-scaling View](https://mlsanity.com/paper/2509.25522/) (2025)
- [NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations](https://mlsanity.com/paper/2511.18793/) (2025)
- [S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation](https://mlsanity.com/paper/2601.18664/) (2026)
- [TBGRecall: A Generative Retrieval Model for E-commerce Recommendation Scenarios](https://mlsanity.com/paper/2508.11977/) (2025)
- [OneLoc: Geo-Aware Generative Recommender Systems for Local Life Service](https://mlsanity.com/paper/2508.14646/) (2025)
- [GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation](https://mlsanity.com/paper/2604.14878/) (2026)
- [Act-With-Think: Chunk Auto-Regressive Modeling for Generative Recommendation](https://mlsanity.com/paper/2506.23643/) (2025)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Models](https://mlsanity.com/paper/2507.06507/) (2025)
- [xLLM Technical Report](https://mlsanity.com/paper/2510.14686/) (2025)
- [When Text-as-Vision Meets Semantic IDs in Generative Recommendation: An Empirical Study](https://mlsanity.com/paper/2601.14697/) (2026)
- [DGenCTR: Towards a Universal Generative Paradigm for Click-Through Rate Prediction via Discrete Diffusion](https://mlsanity.com/paper/2508.14500/) (2025)
- [Masked Diffusion Generative Recommendation](https://mlsanity.com/paper/2601.19501/) (2026)
- [Rethinking Generative Recommender Tokenizer: Recsys-Native Encoding and Semantic Quantization Beyond LLMs](https://mlsanity.com/paper/2602.02338/) (2026)
- [Spend Search Where It Pays: Value-Guided Structured Sampling and Optimization for Generative Recommendation](https://mlsanity.com/paper/2602.10699/) (2026)
- [KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao](https://mlsanity.com/paper/2603.22779/) (2026)
- [HoMer: Addressing Heterogeneities by Modeling Sequential and Set-wise Contexts for CTR Prediction](https://mlsanity.com/paper/2510.11100/) (2025)
- [MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging](https://mlsanity.com/paper/2601.15930/) (2026)
- [LocalSUG: City-Preference-Enhanced LLM for Query Suggestion in Local-Life Services](https://mlsanity.com/paper/2603.04946/) (2026)
- [Learning Variable-Length Tokenization for Generative Recommendation](https://mlsanity.com/paper/2605.17779/) (2026)
- [TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems](https://mlsanity.com/paper/2606.25147/) (2026)
- [EvoRec: Self Evolving Agentic Recommender Systems](https://mlsanity.com/paper/2606.28368/) (2026)
- [RASTP: Representation-Aware Semantic Token Pruning for Generative Recommendation with Semantic Identifiers](https://mlsanity.com/paper/2511.16943/) (2025)
- [PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendation](https://mlsanity.com/paper/2601.16556/) (2026)
- [Trie-Aware Transformers for Generative Recommendation](https://mlsanity.com/paper/2602.21677/) (2026)
- [AgenticRS-Architecture: System Design for Agentic Recommender Systems](https://mlsanity.com/paper/2603.26085/) (2026)
- [Conditional Memory Enhanced Item Representation for Generative Recommendation](https://mlsanity.com/paper/2605.11447/) (2026)

## Abstract (arXiv)

Generative models have recently gained attention in recommendation systems by directly predicting item identifiers from user interaction sequences. However, existing methods suffer from significant information loss due to the separation of stages such as quantization and sequence modeling, hindering their ability to achieve the modeling precision and accuracy of sequential dense retrieval techniques. Integrating generative and dense retrieval methods remains a critical challenge. To address this, we introduce the Cascaded Organized Bi-Represented generAtive retrieval (COBRA) framework, which innovatively integrates sparse semantic IDs and dense vectors through a cascading process. Our method alternates between generating these representations by first generating sparse IDs, which serve as conditions to aid in the generation of dense vectors. End-to-end training enables dynamic refinement of dense representations, capturing both semantic insights and collaborative signals from user-item interactions. During inference, COBRA employs a coarse-to-fine strategy, starting with sparse ID generation and refining them into dense vectors via the generative model. We further propose BeamFusion, an innovative approach combining beam search with nearest neighbor scores to enhance inference flexibility and recommendation diversity. Extensive experiments on public datasets and offline tests validate our method's robustness. Online A/B tests on a real-world advertising platform with over 200 million daily users demonstrate substantial improvements in key metrics, highlighting COBRA's practical advantages.