# PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information

arXiv: https://arxiv.org/abs/2412.06308 · HTML: https://mlsanity.com/paper/2412.06308/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 42 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 8 of 8)

- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling](https://mlsanity.com/paper/2409.12740/) (2024)
- [LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application](https://mlsanity.com/paper/2405.03988/) (2024)
- [RecGPT: Generative Personalized Prompts for Sequential Recommendation via ChatGPT Training Paradigm](https://mlsanity.com/paper/2404.08675/) (2024)
- [TRAWL: External Knowledge-Enhanced Recommendation with LLM Assistance](https://mlsanity.com/paper/2403.06642/) (2024)
- [Multimodal Pre-training Framework for Sequential Recommendation via Contrastive Learning](https://mlsanity.com/paper/2303.11879/) (2023)
- [ID-centric Pre-training for Recommendation](https://mlsanity.com/paper/2405.03562/) (2024)

## Abstract (arXiv)

Real-world recommendation systems commonly offer diverse content scenarios for users to interact with. Considering the enormous number of users in industrial platforms, it is infeasible to utilize a single unified recommendation model to meet the requirements of all scenarios. Usually, separate recommendation pipelines are established for each distinct scenario. This practice leads to challenges in comprehensively grasping users' interests. Recent research endeavors have been made to tackle this problem by pre-training models to encapsulate the overall interests of users. Traditional pre-trained recommendation models mainly capture user interests by leveraging collaborative signals. Nevertheless, a prevalent drawback of these systems is their incapacity to handle long-tail items and cold-start scenarios. With the recent advent of large language models, there has been a significant increase in research efforts focused on exploiting LLMs to extract semantic information for users and items. However, text-based recommendations highly rely on elaborate feature engineering and frequently fail to capture collaborative similarities. To overcome these limitations, we propose a novel pre-training framework for sequential recommendation, termed PRECISE. This framework combines collaborative signals with semantic information. Moreover, PRECISE employs a learning framework that initially models users' comprehensive interests across all recommendation scenarios and subsequently concentrates on the specific interests of target-scene behaviors. We demonstrate that PRECISE precisely captures the entire range of user interests and effectively transfers them to the target interests. Empirical findings reveal that the PRECISE framework attains outstanding performance on both public and industrial datasets.