# Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation

arXiv: https://arxiv.org/abs/2012.07064 · HTML: https://mlsanity.com/paper/2012.07064/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2020.

## Built on by (most cited first, 30 of 42)

- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive](https://mlsanity.com/paper/2105.07342/) (2021)
- [Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods](https://mlsanity.com/paper/2111.04840/) (2021)
- [Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks](https://mlsanity.com/paper/2104.07886/) (2021)
- [LiGNN: Graph Neural Networks at LinkedIn](https://mlsanity.com/paper/2402.11139/) (2024)
- [Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations](https://mlsanity.com/paper/2201.01702/) (2022)
- [Disentangled Contrastive Learning for Social Recommendation](https://mlsanity.com/paper/2208.08723/) (2022)
- [A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications](https://mlsanity.com/paper/2202.07893/) (2022)
- [Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach](https://mlsanity.com/paper/2110.04514/) (2021)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [GraphPro: Graph Pre-training and Prompt Learning for Recommendation](https://mlsanity.com/paper/2311.16716/) (2023)
- [Graph Collaborative Signals Denoising and Augmentation for Recommendation](https://mlsanity.com/paper/2304.03344/) (2023)
- [Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations](https://mlsanity.com/paper/2306.17256/) (2023)
- [A Survey of Imbalanced Learning on Graphs: Problems, Techniques, and Future Directions](https://mlsanity.com/paper/2308.13821/) (2023)
- [Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation](https://mlsanity.com/paper/2306.14462/) (2023)
- [Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training](https://mlsanity.com/paper/2305.07633/) (2023)
- [A Topological Perspective on Demystifying GNN-Based Link Prediction Performance](https://mlsanity.com/paper/2310.04612/) (2023)
- [SoK: Differential Privacy on Graph-Structured Data](https://mlsanity.com/paper/2203.09205/) (2022)
- [MAP: A Model-agnostic Pretraining Framework for Click-through Rate Prediction](https://mlsanity.com/paper/2308.01737/) (2023)
- [Endowing Pre-trained Graph Models with Provable Fairness](https://mlsanity.com/paper/2402.12161/) (2024)
- [EasyRec: Simple yet Effective Language Models for Recommendation](https://mlsanity.com/paper/2408.08821/) (2024)
- [An Adaptive Graph Pre-training Framework for Localized Collaborative Filtering](https://mlsanity.com/paper/2112.07191/) (2021)
- [Doing More with Less: Overcoming Data Scarcity for POI Recommendation via Cross-Region Transfer](https://mlsanity.com/paper/2201.06095/) (2022)
- [Training MLPs on Graphs without Supervision](https://mlsanity.com/paper/2412.03864/) (2024)
- [Cold & Warm Net: Addressing Cold-Start Users in Recommender Systems](https://mlsanity.com/paper/2309.15646/) (2023)
- [BOURNE: Bootstrapped Self-supervised Learning Framework for Unified Graph Anomaly Detection](https://mlsanity.com/paper/2307.15244/) (2023)
- [A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learning](https://mlsanity.com/paper/2402.01440/) (2024)
- [A Multi-Strategy based Pre-Training Method for Cold-Start Recommendation](https://mlsanity.com/paper/2112.02275/) (2021)
- [Neural Graph Matching for Pre-training Graph Neural Networks](https://mlsanity.com/paper/2203.01597/) (2022)
- [Node Duplication Improves Cold-start Link Prediction](https://mlsanity.com/paper/2402.09711/) (2024)

## Abstract (arXiv)

Cold-start problem is a fundamental challenge for recommendation tasks. Despite the recent advances on Graph Neural Networks (GNNs) incorporate the high-order collaborative signal to alleviate the problem, the embeddings of the cold-start users and items aren't explicitly optimized, and the cold-start neighbors are not dealt with during the graph convolution in GNNs. This paper proposes to pre-train a GNN model before applying it for recommendation. Unlike the goal of recommendation, the pre-training GNN simulates the cold-start scenarios from the users/items with sufficient interactions and takes the embedding reconstruction as the pretext task, such that it can directly improve the embedding quality and can be easily adapted to the new cold-start users/items. To further reduce the impact from the cold-start neighbors, we incorporate a self-attention-based meta aggregator to enhance the aggregation ability of each graph convolution step, and an adaptive neighbor sampler to select the effective neighbors according to the feedbacks from the pre-training GNN model. Experiments on three public recommendation datasets show the superiority of our pre-training GNN model against the original GNN models on user/item embedding inference and the recommendation task.