# Zero-Shot Recommendation as Language Modeling

arXiv: https://arxiv.org/abs/2112.04184 · HTML: https://mlsanity.com/paper/2112.04184/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 8 Dec 2021.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Item2Vec: Neural Item Embedding for Collaborative Filtering](https://mlsanity.com/paper/1603.04259/) (2016)
- [Zero-Shot Recommender Systems](https://mlsanity.com/paper/2105.08318/) (2021)
- [What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation](https://mlsanity.com/paper/2007.15356/) (2020)
- [From Zero-Shot Learning to Cold-Start Recommendation](https://mlsanity.com/paper/1906.08511/) (2019)
- [Learning from History and Present: Next-item Recommendation via Discriminatively Exploiting User Behaviors](https://mlsanity.com/paper/1808.01075/) (2018)
- [Zero Shot on the Cold-Start Problem: Model-Agnostic Interest Learning for Recommender Systems](https://mlsanity.com/paper/2108.13592/) (2021)

## Built on by (most cited first, 28 of 28)

- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests Interpretation](https://mlsanity.com/paper/2304.03879/) (2023)
- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [Zero-Shot Next-Item Recommendation using Large Pretrained Language Models](https://mlsanity.com/paper/2304.03153/) (2023)
- [LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking](https://mlsanity.com/paper/2311.02089/) (2023)
- [ChatCounselor: A Large Language Models for Mental Health Support](https://mlsanity.com/paper/2309.15461/) (2023)
- [LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations](https://mlsanity.com/paper/2402.09617/) (2024)
- [Editable User Profiles for Controllable Text Recommendation](https://mlsanity.com/paper/2304.04250/) (2023)
- [Federated Recommendation via Hybrid Retrieval Augmented Generation](https://mlsanity.com/paper/2403.04256/) (2024)
- [Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations](https://mlsanity.com/paper/2306.17256/) (2023)
- [Generative Job Recommendations with Large Language Model](https://mlsanity.com/paper/2307.02157/) (2023)
- [Behavior Alignment: A New Perspective of Evaluating LLM-based Conversational Recommender Systems](https://mlsanity.com/paper/2404.11773/) (2024)
- [A Survey of Graph Prompting Methods: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2303.07275/) (2023)
- [A survey on fairness of large language models in e-commerce: progress, application, and challenge](https://mlsanity.com/paper/2405.13025/) (2024)
- [PAP-REC: Personalized Automatic Prompt for Recommendation Language Model](https://mlsanity.com/paper/2402.00284/) (2024)
- [Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systems](https://mlsanity.com/paper/2309.01188/) (2023)
- [KERAG_R: Knowledge-Enhanced Retrieval-Augmented Generation for Recommendation](https://mlsanity.com/paper/2507.05863/) (2025)
- [Attention Overflow: Language Model Input Blur during Long-Context Missing Items Recommendation](https://mlsanity.com/paper/2407.13481/) (2024)
- [Large Language Models as Conversational Movie Recommenders: A User Study](https://mlsanity.com/paper/2404.19093/) (2024)
- [Large Language Model Driven Recommendation](https://mlsanity.com/paper/2408.10946/) (2024)
- [Instruction-Based Fine-tuning of Open-Source LLMs for Predicting Customer Purchase Behaviors](https://mlsanity.com/paper/2502.15724/) (2025)
- [SocRipple: A Two-Stage Framework for Cold-Start Video Recommendations](https://mlsanity.com/paper/2508.07241/) (2025)
- [Large-scale User Game Lifecycle Representation Learning](https://mlsanity.com/paper/2510.15412/) (2025)
- [Towards Efficient Hypergraph and Multi-LLM Agent Recommender Systems](https://mlsanity.com/paper/2512.06590/) (2025)
- [GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models](https://mlsanity.com/paper/2607.24764/) (2026)
- [TEAR: Table Extraction with Attribute Recommendation from Texts via Large Language Models](https://mlsanity.com/paper/2609.15205/) (2026)

## Abstract (arXiv)

Recommendation is the task of ranking items (e.g. movies or products) according to individual user needs. Current systems rely on collaborative filtering and content-based techniques, which both require structured training data. We propose a framework for recommendation with off-the-shelf pretrained language models (LM) that only used unstructured text corpora as training data. If a user u liked textitMatrix and textitInception, we construct a textual prompt, e.g. textit"Movies like Matrix, Inception, <m>" to estimate the affinity between u and m with LM likelihood. We motivate our idea with a corpus analysis, evaluate several prompt structures, and we compare LM-based recommendation with standard matrix factorization trained on different data regimes. The code for our experiments is publicly available (https://colab.research.google.com/drive/1f1mlZ-FGaLGdo5rPzxf3vemKllbh2esT?usp=sharing).