# Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems

arXiv: https://arxiv.org/abs/2302.03735 · HTML: https://mlsanity.com/paper/2302.03735/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2023.

## Built on by (most cited first, 30 of 45)

- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach](https://mlsanity.com/paper/2305.07001/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs](https://mlsanity.com/paper/2407.10834/) (2024)
- [UP5: Unbiased Foundation Model for Fairness-aware Recommendation](https://mlsanity.com/paper/2305.12090/) (2023)
- [A Large Language Model Enhanced Conversational Recommender System](https://mlsanity.com/paper/2308.06212/) (2023)
- [HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis](https://mlsanity.com/paper/2305.18226/) (2023)
- [Understanding Biases in ChatGPT-based Recommender Systems: Provider Fairness, Temporal Stability, and Recency](https://mlsanity.com/paper/2401.10545/) (2024)
- [A Survey on Diffusion Models for Recommender Systems](https://mlsanity.com/paper/2409.05033/) (2024)
- [Personalized LoRA for Human-Centered Text Understanding](https://mlsanity.com/paper/2403.06208/) (2024)
- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [Knowledge Prompt-tuning for Sequential Recommendation](https://mlsanity.com/paper/2308.08459/) (2023)
- [A Survey on Large Language Models for Personalized and Explainable Recommendations](https://mlsanity.com/paper/2311.12338/) (2023)
- [LettinGo: Explore User Profile Generation for Recommendation System](https://mlsanity.com/paper/2506.18309/) (2025)
- [A Survey of Graph Prompting Methods: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2303.07275/) (2023)
- [LLaMA-E: Empowering E-commerce Authoring with Object-Interleaved Instruction Following](https://mlsanity.com/paper/2308.04913/) (2023)
- [LightLM: A Lightweight Deep and Narrow Language Model for Generative Recommendation](https://mlsanity.com/paper/2310.17488/) (2023)
- [A Survey on Large Language Models in Multimodal Recommender Systems](https://mlsanity.com/paper/2505.09777/) (2025)
- [Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language Modeling](https://mlsanity.com/paper/2309.10435/) (2023)
- [Collaborative Cross-modal Fusion with Large Language Model for Recommendation](https://mlsanity.com/paper/2408.08564/) (2024)
- [Graph Prompting for Graph Learning Models: Recent Advances and Future Directions](https://mlsanity.com/paper/2506.08326/) (2025)
- [Enhancing ID-based Recommendation with Large Language Models](https://mlsanity.com/paper/2411.02041/) (2024)
- [X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2504.20859/) (2025)
- [Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?](https://mlsanity.com/paper/2602.13626/) (2026)
- [Opportunities and Challenges of Applying Large Language Models in Building Energy Efficiency and Decarbonization Studies: An Exploratory Overview](https://mlsanity.com/paper/2312.11701/) (2023)

## Abstract (arXiv)

The emergence of Pre-trained Language Models (PLMs) has achieved tremendous success in the field of Natural Language Processing (NLP) by learning universal representations on large corpora in a self-supervised manner. The pre-trained models and the learned representations can be beneficial to a series of downstream NLP tasks. This training paradigm has recently been adapted to the recommendation domain and is considered a promising approach by both academia and industry. In this paper, we systematically investigate how to extract and transfer knowledge from pre-trained models learned by different PLM-related training paradigms to improve recommendation performance from various perspectives, such as generality, sparsity, efficiency and effectiveness. Specifically, we propose a comprehensive taxonomy to divide existing PLM-based recommender systems w.r.t. their training strategies and objectives. Then, we analyze and summarize the connection between PLM-based training paradigms and different input data types for recommender systems. Finally, we elaborate on open issues and future research directions in this vibrant field.