# Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations

arXiv: https://arxiv.org/abs/2306.17256 · HTML: https://mlsanity.com/paper/2306.17256/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 68 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach](https://mlsanity.com/paper/2305.07001/) (2023)
- [M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems](https://mlsanity.com/paper/2205.08084/) (2022)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems](https://mlsanity.com/paper/1803.03467/) (2018)
- [MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation](https://mlsanity.com/paper/1908.00413/) (2019)
- [Self-Supervised Learning for Recommender Systems: A Survey](https://mlsanity.com/paper/2203.15876/) (2022)
- [Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering](https://mlsanity.com/paper/2202.13296/) (2022)
- [Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning](https://mlsanity.com/paper/2206.09363/) (2022)
- [Personalized Prompt Learning for Explainable Recommendation](https://mlsanity.com/paper/2202.07371/) (2022)
- [Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation](https://mlsanity.com/paper/1901.08907/) (2019)
- [Optimized Cost per Click in Taobao Display Advertising](https://mlsanity.com/paper/1703.02091/) (2017)

## Built on by (most cited first, 15 of 15)

- [Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering](https://mlsanity.com/paper/2403.19631/) (2024)
- [A Survey on Large Language Models for Personalized and Explainable Recommendations](https://mlsanity.com/paper/2311.12338/) (2023)
- [Large Language Model with Graph Convolution for Recommendation](https://mlsanity.com/paper/2402.08859/) (2024)
- [RAMO: Retrieval-Augmented Generation for Enhancing MOOCs Recommendations](https://mlsanity.com/paper/2407.04925/) (2024)
- [Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation](https://mlsanity.com/paper/2503.05493/) (2025)
- [The Pitfalls of Growing Group Complexity: LLMs and Social Choice-Based Aggregation for Group Recommendations](https://mlsanity.com/paper/2505.05016/) (2025)
- [Optimizing Novelty of Top-k Recommendations using Large Language Models and Reinforcement Learning](https://mlsanity.com/paper/2406.14169/) (2024)
- [Consistent Explainers or Unreliable Narrators? Understanding LLM-generated Group Recommendations](https://mlsanity.com/paper/2507.13705/) (2025)
- [Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendation](https://mlsanity.com/paper/2510.05125/) (2025)
- [MixLM: High-Throughput and Effective LLM Ranking via Text-Embedding Mix-Interaction](https://mlsanity.com/paper/2512.07846/) (2025)
- [Semantic Search At LinkedIn](https://mlsanity.com/paper/2602.07309/) (2026)
- [Prompt Transfer for Dual-Aspect Cross Domain Cognitive Diagnosis](https://mlsanity.com/paper/2412.05004/) (2024)
- [MDiffFR: Modality-Guided Diffusion Generation for Cold-start Items in Federated Recommendation](https://mlsanity.com/paper/2512.24715/) (2025)
- [Towards Sustainable Growth: A Multi-Value-Aware Retrieval Framework for E-Commerce Search](https://mlsanity.com/paper/2605.17994/) (2026)
- [Consensus vs. Dissent: Dynamic LLM Modeling of Subjective Preferences in Group Recommenders](https://mlsanity.com/paper/2607.10235/) (2026)

## Abstract (arXiv)

Recommendation systems help users find matched items based on their previous behaviors. Personalized recommendation becomes challenging in the absence of historical user-item interactions, a practical problem for startups known as the system cold-start recommendation. While existing research addresses cold-start issues for either users or items, we still lack solutions for system cold-start scenarios. To tackle the problem, we propose PromptRec, a simple but effective approach based on in-context learning of language models, where we transform the recommendation task into the sentiment analysis task on natural language containing user and item profiles. However, this naive approach heavily relies on the strong in-context learning ability emerged from large language models, which could suffer from significant latency for online recommendations. To solve the challenge, we propose to enhance small language models for recommender systems with a data-centric pipeline, which consists of: (1) constructing a refined corpus for model pre-training; (2) constructing a decomposed prompt template via prompt pre-training. They correspond to the development of training data and inference data, respectively. The pipeline is supported by a theoretical framework that formalizes the connection between in-context recommendation and language modeling. To evaluate our approach, we introduce a cold-start recommendation benchmark, and the results demonstrate that the enhanced small language models can achieve comparable cold-start recommendation performance to that of large models with only 17% of the inference time. To the best of our knowledge, this is the first study to tackle the system cold-start recommendation problem. We believe our findings will provide valuable insights for future works. The benchmark and implementations are available at https://github.com/JacksonWuxs/PromptRec.