# UPRec: User-Aware Pre-training for Recommender Systems

arXiv: https://arxiv.org/abs/2102.10989 · HTML: https://mlsanity.com/paper/2102.10989/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2021.

## Built on by (most cited first, 16 of 16)

- [Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach](https://mlsanity.com/paper/2305.07001/) (2023)
- [Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders](https://mlsanity.com/paper/2210.12316/) (2022)
- [MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation](https://mlsanity.com/paper/2308.11175/) (2023)
- [Multi-view Multi-behavior Contrastive Learning in Recommendation](https://mlsanity.com/paper/2203.10576/) (2022)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [KEEP: An Industrial Pre-Training Framework for Online Recommendation via Knowledge Extraction and Plugging](https://mlsanity.com/paper/2208.10174/) (2022)
- [A Comprehensive Survey on Self-Supervised Learning for Recommendation](https://mlsanity.com/paper/2404.03354/) (2024)
- [Learning Large-scale Universal User Representation with Sparse Mixture of Experts](https://mlsanity.com/paper/2207.04648/) (2022)
- [Denoising Pre-Training and Customized Prompt Learning for Efficient Multi-Behavior Sequential Recommendation](https://mlsanity.com/paper/2408.11372/) (2024)
- [Spatial Autoregressive Coding for Graph Neural Recommendation](https://mlsanity.com/paper/2205.09489/) (2022)
- [Generative Pretraining at Scale: Transformer-Based Encoding of Transactional Behavior for Fraud Detection](https://mlsanity.com/paper/2312.14406/) (2023)
- [Prompt Tuning for Item Cold-start Recommendation](https://mlsanity.com/paper/2412.18082/) (2024)
- [Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendation](https://mlsanity.com/paper/2510.05125/) (2025)
- [A Social-aware Gaussian Pre-trained Model for Effective Cold-start Recommendation](https://mlsanity.com/paper/2311.15790/) (2023)
- [PPM : A Pre-trained Plug-in Model for Click-through Rate Prediction](https://mlsanity.com/paper/2403.10049/) (2024)
- [User Representation via Cross Multi-source Behavior Pre-training for Mobile Games](https://mlsanity.com/paper/2609.01057/) (2026)

## Abstract (arXiv)

Existing sequential recommendation methods rely on large amounts of training data and usually suffer from the data sparsity problem. To tackle this, the pre-training mechanism has been widely adopted, which attempts to leverage large-scale data to perform self-supervised learning and transfer the pre-trained parameters to downstream tasks. However, previous pre-trained models for recommendation focus on leverage universal sequence patterns from user behaviour sequences and item information, whereas ignore capturing personalized interests with the heterogeneous user information, which has been shown effective in contributing to personalized recommendation. In this paper, we propose a method to enhance pre-trained models with heterogeneous user information, called User-aware Pre-training for Recommendation (UPRec). Specifically, UPRec leverages the user attributes andstructured social graphs to construct self-supervised objectives in the pre-training stage and proposes two user-aware pre-training tasks. Comprehensive experimental results on several real-world large-scale recommendation datasets demonstrate that UPRec can effectively integrate user information into pre-trained models and thus provide more appropriate recommendations for users.