mlsanityData to 14 Oct 2026

Multi-task Item-attribute Graph Pre-training for Strict Cold-start Item Recommendation

arXiv 2306.14462 ↗First posted 26 Jun 2023cs.IR cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Recommendation systems suffer in the strict cold-start (SCS) scenario, where the user-item interactions are entirely unavailable. The ID-based approaches completely fail to work. Cold-start recommenders, on the other hand, leverage item contents to map the new items to the existing ones. However, the existing SCS recommenders explore item contents in coarse-grained manners that introduce noise or information loss. Moreover, informative data sources other than item contents, such as users' purchase sequences and review texts, are ignored. We explore the role of the fine-grained item attributes in bridging the gaps between the existing and the SCS items and pre-train a knowledgeable item-attribute graph for SCS item recommendation. Our proposed framework, ColdGPT, models item-attribute correlations into an item-attribute graph by extracting fine-grained attributes from item contents. ColdGPT then transfers knowledge into the item-attribute graph from various available data sources, i.e., item contents, historical purchase sequences, and review texts of the existing items, via multi-task learning. To facilitate the positive transfer, ColdGPT designs submodules according to the natural forms of the data sources and coordinates the multiple pre-training tasks via unified alignment-and-uniformity losses. Our pre-trained item-attribute graph acts as an implicit, extendable item embedding matrix, which enables the SCS item embeddings to be easily acquired by inserting these items and propagating their attributes' embeddings. We carefully process three public datasets, i.e., Yelp, Amazon-home, and Amazon-sports, to guarantee the SCS setting for evaluation. Extensive experiments show that ColdGPT consistently outperforms the existing SCS recommenders by large margins and even surpasses models that are pre-trained on 75-224 times more, cross-domain data on two out of four datasets.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 61 of its 80 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

61 of 80 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2017Graph Attention Networkscited by 8,923
  8. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  10. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  11. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  12. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
38 more
  1. 2018Self-Attentive Sequential Recommendationcited by 1,184
  2. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,139
  3. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  4. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  5. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  6. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  7. 2016Wide & Deep Learning for Recommender Systemscited by 747
  8. 2019Graph Neural Networks for Social Recommendationcited by 714
  9. 2019Neural Graph Collaborative Filteringcited by 661
  10. 2015LINE: Large-scale Information Network Embeddingcited by 582
  11. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  12. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
  13. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  14. 2019KGAT: Knowledge Graph Attention Network for Recommendationcited by 343
  15. 2020Self-supervised Graph Learning for Recommendationcited by 305
  16. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 303
  17. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 201
  18. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  19. 2017Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networkscited by 126
  20. 2023Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisitedcited by 124
  21. 2019Knowledge Graph Convolutional Networks for Recommender Systemscited by 121
  22. 2019MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendationcited by 121
  23. 2022Towards Universal Sequence Representation Learning for Recommender Systemscited by 121
  24. 2019Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferencescited by 87
  25. 2021RotoGrad: Gradient Homogenization in Multitask Learningcited by 83
  26. 2021Contrastive Learning for Cold-Start Recommendationcited by 79
  27. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 67
  28. 2019Warm Up Cold-start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddingscited by 66
  29. 2022Do Current Multi-Task Optimization Methods in Deep Learning Even Help?cited by 60
  30. 2021Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoodscited by 58
  31. 2022Towards Representation Alignment and Uniformity in Collaborative Filteringcited by 57
  32. 2021Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformercited by 45
  33. 2023Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspectivecited by 43
  34. 2020Pre-Training Graph Neural Networks for Cold-Start Users and Items Representationcited by 42
  35. 2020MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendationcited by 39
  36. 2021ConsisRec: Enhancing GNN for Social Recommendation via Consistent Neighbor Aggregationcited by 39
  37. 2021Learning to Warm Up Cold Item Embeddings for Cold-start Recommendation with Meta Scaling and Shifting Networkscited by 37
  38. 2021Task-adaptive Neural Process for User Cold-Start Recommendationcited by 32

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 2; 2026: 3.

13 papers.

Sort:

By area:

  1. 2023Data Scarcity in Recommendation Systems: A Surveycited by 17
  2. 2024Taxonomy-Guided Zero-Shot Recommendations with LLMscited by 15
  3. 2024A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarioscited by 13
  4. 2024A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learningcited by 6
  5. 2024Bridging the User-side Knowledge Gap in Knowledge-aware Recommendations with Large Language Modelscited by 6
  6. 2023Unified Pretraining for Recommendation via Task Hypergraphscited by 4
  7. 2026The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordinationcited by 3
  8. 2024Fair Augmentation for Graph Collaborative Filteringcited by 1
  9. 2024Prompt Tuning for Item Cold-start Recommendationcited by 1
  10. 2025Personalized Fashion Recommendation with Image Attributes and Aesthetics Assessmentcited by 1
  11. 2026No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrievalcited by 1
  12. 2025Unifying Inductive, Cross-Domain, and Multimodal Learning for Robust and Generalizable Recommendationcited by 0
1 more
  1. 2026Dual-Attention Convolution Experts for Sparse Tensor Completioncited by 0