# RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation

arXiv: https://arxiv.org/abs/2509.03131 · HTML: https://mlsanity.com/paper/2509.03131/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 54 references that are in the map.
- First posted on arXiv 3 Sep 2025.

## Builds on (most cited first, 30 of 38)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems](https://mlsanity.com/paper/2205.08084/) (2022)
- [KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems](https://mlsanity.com/paper/2202.10842/) (2022)
- [Fine-Grained Spoiler Detection from Large-Scale Review Corpora](https://mlsanity.com/paper/1905.13416/) (2019)
- [GenRec: Large Language Model for Generative Recommendation](https://mlsanity.com/paper/2307.00457/) (2023)
- [EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration](https://mlsanity.com/paper/2406.14017/) (2024)
- [BARS: Towards Open Benchmarking for Recommender Systems](https://mlsanity.com/paper/2205.09626/) (2022)
- [A Content-Driven Micro-Video Recommendation Dataset at Scale](https://mlsanity.com/paper/2309.15379/) (2023)
- [Hierarchical Fashion Graph Network for Personalized Outfit Recommendation](https://mlsanity.com/paper/2005.12566/) (2020)
- [ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction](https://mlsanity.com/paper/2310.09234/) (2023)
- [CrossCBR: Cross-view Contrastive Learning for Bundle Recommendation](https://mlsanity.com/paper/2206.00242/) (2022)
- [CoST: Contrastive Quantization based Semantic Tokenization for Generative Recommendation](https://mlsanity.com/paper/2404.14774/) (2024)
- [An Image Dataset for Benchmarking Recommender Systems with Raw Pixels](https://mlsanity.com/paper/2309.06789/) (2023)
- [Content-Based Collaborative Generation for Recommender Systems](https://mlsanity.com/paper/2403.18480/) (2024)

## Built on by (most cited first, 2 of 2)

- [Generative Reasoning Recommendation via LLMs](https://mlsanity.com/paper/2510.20815/) (2025)
- [MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging](https://mlsanity.com/paper/2601.15930/) (2026)

## Abstract (arXiv)

Recent advances in LLM-based recommendation have shown promise, yet their cross-domain generalization is hindered by a fundamental mismatch between language-centric pretraining and the recommendation task. Existing methods, relying on language-level knowledge, fail to capture dynamic, item-level user interests across domains. To bridge this gap, we propose RecBase, a domain-agnostic foundational model pretrained with a recommendation-oriented objective. RecBase leverages a large-scale, heterogeneous, cross-domain corpus with unified textual representations and feature mappings to enhance cross-domain generalization. To further align item semantics across domains, we introduce a unified item tokenizer that encodes items into hierarchical concept identifiers, enabling structured representation and efficient vocabulary sharing. The model is trained using an autoregressive objective to capture complex item-level sequential patterns. On eight real-world datasets, our 1.5B-parameter model matches or surpasses the performance of LLM baselines up to 7B parameters in zero-shot and cross-domain recommendation tasks.