# Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training

arXiv: https://arxiv.org/abs/2305.07633 · HTML: https://mlsanity.com/paper/2305.07633/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 54 references that are in the map.
- First posted on arXiv 12 May 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning](https://mlsanity.com/paper/2105.09111/) (2021)
- [Towards Universal Sequence Representation Learning for Recommender Systems](https://mlsanity.com/paper/2206.05941/) (2022)
- [Inferring Networks of Substitutable and Complementary Products](https://mlsanity.com/paper/1506.08839/) (2015)
- [SimGNN: A Neural Network Approach to Fast Graph Similarity Computation](https://mlsanity.com/paper/1808.05689/) (2018)
- [NAIS: Neural Attentive Item Similarity Model for Recommendation](https://mlsanity.com/paper/1809.07053/) (2018)
- [Learning-based Efficient Graph Similarity Computation via Multi-Scale Convolutional Set Matching](https://mlsanity.com/paper/1809.04440/) (2018)
- [Zero-Shot Recommender Systems](https://mlsanity.com/paper/2105.08318/) (2021)
- [Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation](https://mlsanity.com/paper/2012.07064/) (2020)
- [Contrastive Meta Learning with Behavior Multiplicity for Recommendation](https://mlsanity.com/paper/2202.08523/) (2022)
- [Product Knowledge Graph Embedding for E-commerce](https://mlsanity.com/paper/1911.12481/) (2019)
- [Try This Instead: Personalized and Interpretable Substitute Recommendation](https://mlsanity.com/paper/2005.09344/) (2020)
- [AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types](https://mlsanity.com/paper/2006.13473/) (2020)
- [From Zero-Shot Learning to Cold-Start Recommendation](https://mlsanity.com/paper/1906.08511/) (2019)
- [Cross-Market Product Recommendation](https://mlsanity.com/paper/2109.05929/) (2021)

## Built on by (most cited first, 12 of 12)

- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs](https://mlsanity.com/paper/2312.03731/) (2023)
- [Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation](https://mlsanity.com/paper/2306.14462/) (2023)
- [Scaling Sequential Recommendation Models with Transformers](https://mlsanity.com/paper/2412.07585/) (2024)
- [A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learning](https://mlsanity.com/paper/2402.01440/) (2024)
- [Towards Explainable Temporal User Profiling with LLMs](https://mlsanity.com/paper/2505.00886/) (2025)
- [Fully Hyperbolic Rotation for Knowledge Graph Embedding](https://mlsanity.com/paper/2411.03622/) (2024)
- [Temporal User Profiling with LLMs: Balancing Short-Term and Long-Term Preferences for Recommendations](https://mlsanity.com/paper/2508.08454/) (2025)
- [A Survey of Reasoning for Substitution Relationships: Definitions, Methods, and Directions](https://mlsanity.com/paper/2404.08687/) (2024)
- [A Zero-shot Learning Method Based on Large Language Models for Multi-modal Knowledge Graph Embedding](https://mlsanity.com/paper/2503.07202/) (2025)
- [A Scenario-Oriented Survey of Federated Recommender Systems: Techniques, Challenges, and Future Directions](https://mlsanity.com/paper/2508.19620/) (2025)
- [Knowledge Graphs Meet Graph Neural Networks: A Comprehensive Survey](https://mlsanity.com/paper/2607.09666/) (2026)

## Abstract (arXiv)

Existing recommender systems face difficulties with zero-shot items, i.e. items that have no historical interactions with users during the training stage. Though recent works extract universal item representation via pre-trained language models (PLMs), they ignore the crucial item relationships. This paper presents a novel paradigm for the Zero-Shot Item-based Recommendation (ZSIR) task, which pre-trains a model on product knowledge graph (PKG) to refine the item features from PLMs. We identify three challenges for pre-training PKG, which are multi-type relations in PKG, semantic divergence between item generic information and relations and domain discrepancy from PKG to downstream ZSIR task. We address the challenges by proposing four pre-training tasks and novel task-oriented adaptation (ToA) layers. Moreover, this paper discusses how to fine-tune the model on new recommendation task such that the ToA layers are adapted to ZSIR task. Comprehensive experiments on 18 markets dataset are conducted to verify the effectiveness of the proposed model in both knowledge prediction and ZSIR task.