# RecGPT: Generative Pre-training for Text-based Recommendation

arXiv: https://arxiv.org/abs/2405.12715 · HTML: https://mlsanity.com/paper/2405.12715/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [DKN: Deep Knowledge-Aware Network for News Recommendation](https://mlsanity.com/paper/1801.08284/) (2018)
- [GenRec: Large Language Model for Generative Recommendation](https://mlsanity.com/paper/2307.00457/) (2023)
- [Generating Personalized Recipes from Historical User Preferences](https://mlsanity.com/paper/1909.00105/) (2019)
- [VIP5: Towards Multimodal Foundation Models for Recommendation](https://mlsanity.com/paper/2305.14302/) (2023)
- [HotelRec: a Novel Very Large-Scale Hotel Recommendation Dataset](https://mlsanity.com/paper/2002.06854/) (2020)
- [Learning to Recommend Items to Wikidata Editors](https://mlsanity.com/paper/2107.06423/) (2021)

## Built on by (most cited first, 9 of 9)

- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [Causality-Enhanced Behavior Sequence Modeling in LLMs for Personalized Recommendation](https://mlsanity.com/paper/2410.22809/) (2024)
- [Can LLMs Outshine Conventional Recommenders? A Comparative Evaluation](https://mlsanity.com/paper/2503.05493/) (2025)
- [CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems](https://mlsanity.com/paper/2506.19993/) (2025)
- [Lightweight Fairness for LLM-Based Recommendations via Kernelized Projection and Gated Adapters](https://mlsanity.com/paper/2603.23780/) (2026)
- [SPECTRA: Revealing the Full Spectrum of User Preferences via Distributional LLM Inference](https://mlsanity.com/paper/2509.24189/) (2025)
- [Give Users the Wheel: Towards Promptable Recommendation Paradigm](https://mlsanity.com/paper/2602.18929/) (2026)
- [RPCBench: A Benchmark for Proactive Premise Critique in LLM-based Recommendation](https://mlsanity.com/paper/2609.00918/) (2026)

## Abstract (arXiv)

We present the first domain-adapted and fully-trained large language model, RecGPT-7B, and its instruction-following variant, RecGPT-7B-Instruct, for text-based recommendation. Experimental results on rating prediction and sequential recommendation tasks show that our model, RecGPT-7B-Instruct, outperforms previous strong baselines. We are releasing our RecGPT models as well as their pre-training and fine-tuning datasets to facilitate future research and downstream applications in text-based recommendation. Public "huggingface" links to our RecGPT models and datasets are available at: https://github.com/VinAIResearch/RecGPT