# Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systems

arXiv: https://arxiv.org/abs/2309.01188 · HTML: https://mlsanity.com/paper/2309.01188/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation](https://mlsanity.com/paper/1908.00413/) (2019)
- [Causal Intervention for Leveraging Popularity Bias in Recommendation](https://mlsanity.com/paper/2105.06067/) (2021)
- [CoNet: Collaborative Cross Networks for Cross-Domain Recommendation](https://mlsanity.com/paper/1804.06769/) (2018)
- [Contrastive Learning for Cold-Start Recommendation](https://mlsanity.com/paper/2107.05315/) (2021)
- [Session-based Social Recommendation via Dynamic Graph Attention Networks](https://mlsanity.com/paper/1902.09362/) (2019)
- [DDTCDR: Deep Dual Transfer Cross Domain Recommendation](https://mlsanity.com/paper/1910.05189/) (2019)
- [Zero-Shot Recommender Systems](https://mlsanity.com/paper/2105.08318/) (2021)
- [CATN: Cross-Domain Recommendation for Cold-Start Users via Aspect Transfer Network](https://mlsanity.com/paper/2005.10549/) (2020)
- [Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking](https://mlsanity.com/paper/1707.05176/) (2017)
- [MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendation](https://mlsanity.com/paper/2007.03183/) (2020)
- [The Connection Between Popularity Bias, Calibration, and Fairness in Recommendation](https://mlsanity.com/paper/2008.09273/) (2020)
- [Zero-Shot Recommendation as Language Modeling](https://mlsanity.com/paper/2112.04184/) (2021)
- [Transfer-Meta Framework for Cross-domain Recommendation to Cold-Start Users](https://mlsanity.com/paper/2105.04785/) (2021)
- [Cross-Domain Recommendation to Cold-Start Users via Variational Information Bottleneck](https://mlsanity.com/paper/2203.16863/) (2022)
- [Collaborative Filtering with Attribution Alignment for Review-based Non-overlapped Cross Domain Recommendation](https://mlsanity.com/paper/2202.04920/) (2022)

## Built on by (most cited first, 3 of 3)

- [LLM-RecG: A Semantic Bias-Aware Framework for Zero-Shot Sequential Recommendation](https://mlsanity.com/paper/2501.19232/) (2025)
- [Differentially Private Relational Learning with Entity-level Privacy Guarantees](https://mlsanity.com/paper/2506.08347/) (2025)
- [RecPFN: Prior-Fitted Networks for In-Context-Based Recommendations](https://mlsanity.com/paper/2608.19735/) (2026)

## Abstract (arXiv)

Modern neural collaborative filtering techniques are critical to the success of e-commerce, social media, and content-sharing platforms. However, despite technical advances -- for every new application domain, we need to train an NCF model from scratch. In contrast, pre-trained vision and language models are routinely applied to diverse applications directly (zero-shot) or with limited fine-tuning. Inspired by the impact of pre-trained models, we explore the possibility of pre-trained recommender models that support building recommender systems in new domains, with minimal or no retraining, without the use of any auxiliary user or item information. Zero-shot recommendation without auxiliary information is challenging because we cannot form associations between users and items across datasets when there are no overlapping users or items. Our fundamental insight is that the statistical characteristics of the user-item interaction matrix are universally available across different domains and datasets. Thus, we use the statistical characteristics of the user-item interaction matrix to identify dataset-independent representations for users and items. We show how to learn universal (i.e., supporting zero-shot adaptation without user or item auxiliary information) representations for nodes and edges from the bipartite user-item interaction graph. We learn representations by exploiting the statistical properties of the interaction data, including user and item marginals, and the size and density distributions of their clusters.