# An Unified Search and Recommendation Foundation Model for Cold-Start Scenario

arXiv: https://arxiv.org/abs/2309.08939 · HTML: https://mlsanity.com/paper/2309.08939/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 16 Sep 2023.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems](https://mlsanity.com/paper/2205.08084/) (2022)
- [One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction](https://mlsanity.com/paper/2101.11427/) (2021)
- [A Model-Agnostic Causal Learning Framework for Recommendation using Search Data](https://mlsanity.com/paper/2202.04514/) (2022)
- [When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendation](https://mlsanity.com/paper/2305.10822/) (2023)
- [USER: A Unified Information Search and Recommendation Model based on Integrated Behavior Sequence](https://mlsanity.com/paper/2109.15012/) (2021)
- [Joint Modeling and Optimization of Search and Recommendation](https://mlsanity.com/paper/1807.05631/) (2018)
- [Infer Implicit Contexts in Real-time Online-to-Offline Recommendation](https://mlsanity.com/paper/1907.04924/) (2019)
- [Prototypical Contrastive Learning and Adaptive Interest Selection for Candidate Generation in Recommendations](https://mlsanity.com/paper/2211.12893/) (2022)

## Built on by (most cited first, 12 of 12)

- [When Search Engine Services meet Large Language Models: Visions and Challenges](https://mlsanity.com/paper/2407.00128/) (2024)
- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [Bringing Generative AI to Adaptive Learning in Education](https://mlsanity.com/paper/2402.14601/) (2024)
- [Exploring User Retrieval Integration towards Large Language Models for Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2406.03085/) (2024)
- [LLM4MSR: An LLM-Enhanced Paradigm for Multi-Scenario Recommendation](https://mlsanity.com/paper/2406.12529/) (2024)
- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios](https://mlsanity.com/paper/2409.17864/) (2024)
- [STAR: A Simple Training-free Approach for Recommendations using Large Language Models](https://mlsanity.com/paper/2410.16458/) (2024)
- [Navigating the Future of Federated Recommendation Systems with Foundation Models](https://mlsanity.com/paper/2406.00004/) (2024)
- [SEMINAR: Search Enhanced Multi-modal Interest Network and Approximate Retrieval for Lifelong Sequential Recommendation](https://mlsanity.com/paper/2407.10714/) (2024)
- [Leveraging Multimodal Data and Side Users for Diffusion Cross-Domain Recommendation](https://mlsanity.com/paper/2507.04000/) (2025)
- [Progressive Alignment of Recommender Foundation Model through Multi-Phase Post-Training](https://mlsanity.com/paper/2608.06792/) (2026)

## Abstract (arXiv)

In modern commercial search engines and recommendation systems, data from multiple domains is available to jointly train the multi-domain model. Traditional methods train multi-domain models in the multi-task setting, with shared parameters to learn the similarity of multiple tasks, and task-specific parameters to learn the divergence of features, labels, and sample distributions of individual tasks. With the development of large language models, LLM can extract global domain-invariant text features that serve both search and recommendation tasks. We propose a novel framework called S&R Multi-Domain Foundation, which uses LLM to extract domain invariant features, and Aspect Gating Fusion to merge the ID feature, domain invariant text features and task-specific heterogeneous sparse features to obtain the representations of query and item. Additionally, samples from multiple search and recommendation scenarios are trained jointly with Domain Adaptive Multi-Task module to obtain the multi-domain foundation model. We apply the S&R Multi-Domain foundation model to cold start scenarios in the pretrain-finetune manner, which achieves better performance than other SOTA transfer learning methods. The S&R Multi-Domain Foundation model has been successfully deployed in Alipay Mobile Application's online services, such as content query recommendation and service card recommendation, etc.