# Unified Pretraining for Recommendation via Task Hypergraphs

arXiv: https://arxiv.org/abs/2310.13286 · HTML: https://mlsanity.com/paper/2310.13286/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems](https://mlsanity.com/paper/2205.08084/) (2022)
- [Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation](https://mlsanity.com/paper/2101.06448/) (2021)
- [UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation](https://mlsanity.com/paper/2110.15114/) (2021)
- [Hypergraph Contrastive Collaborative Filtering](https://mlsanity.com/paper/2204.12200/) (2022)
- [Graph Learning based Recommender Systems: A Review](https://mlsanity.com/paper/2105.06339/) (2021)
- [Towards Representation Alignment and Uniformity in Collaborative Filtering](https://mlsanity.com/paper/2206.12811/) (2022)
- [Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation](https://mlsanity.com/paper/2306.14462/) (2023)
- [Group Identification via Transitional Hypergraph Convolution with Cross-view Self-supervised Learning](https://mlsanity.com/paper/2308.08620/) (2023)
- [Ranking-based Group Identification via Factorized Attention on Social Tripartite Graph](https://mlsanity.com/paper/2211.01830/) (2022)
- [Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation](https://mlsanity.com/paper/2301.12197/) (2023)

## Built on by (most cited first, 4 of 4)

- [Taxonomy-Guided Zero-Shot Recommendations with LLMs](https://mlsanity.com/paper/2406.14043/) (2024)
- [Heterogeneous Hypergraph Embedding for Recommendation Systems](https://mlsanity.com/paper/2407.03665/) (2024)
- [Handling Heterophily in Recommender Systems with Wavelet Hypergraph Diffusion](https://mlsanity.com/paper/2501.14399/) (2025)
- [Hypergraph Diffusion for High-Order Recommender Systems](https://mlsanity.com/paper/2501.16722/) (2025)

## Abstract (arXiv)

Although pretraining has garnered significant attention and popularity in recent years, its application in graph-based recommender systems is relatively limited. It is challenging to exploit prior knowledge by pretraining in widely used ID-dependent datasets. On one hand, user-item interaction history in one dataset can hardly be transferred to other datasets through pretraining, where IDs are different. On the other hand, pretraining and finetuning on the same dataset leads to a high risk of overfitting. In this paper, we propose a novel multitask pretraining framework named Unified Pretraining for Recommendation via Task Hypergraphs. For a unified learning pattern to handle diverse requirements and nuances of various pretext tasks, we design task hypergraphs to generalize pretext tasks to hyperedge prediction. A novel transitional attention layer is devised to discriminatively learn the relevance between each pretext task and recommendation. Experimental results on three benchmark datasets verify the superiority of UPRTH. Additional detailed investigations are conducted to demonstrate the effectiveness of the proposed framework.