# Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning

arXiv: https://arxiv.org/abs/2503.24289 · HTML: https://mlsanity.com/paper/2503.24289/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 88 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Builds on (most cited first, 30 of 70)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach](https://mlsanity.com/paper/2305.07001/) (2023)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems](https://mlsanity.com/paper/2205.08084/) (2022)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)

## Built on by (most cited first, 30 of 33)

- [An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents](https://mlsanity.com/paper/2505.15117/) (2025)
- [SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs](https://mlsanity.com/paper/2509.20758/) (2025)
- [Customer-R1: Personalized Simulation of Human Behaviors via RL-based LLM Agent in Online Shopping](https://mlsanity.com/paper/2510.07230/) (2025)
- [Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewards](https://mlsanity.com/paper/2505.02686/) (2025)
- [Training LLMs for EHR-Based Reasoning Tasks via Reinforcement Learning](https://mlsanity.com/paper/2505.24105/) (2025)
- [R2ec: Towards Large Recommender Models with Reasoning](https://mlsanity.com/paper/2505.16994/) (2025)
- [Slow Thinking for Sequential Recommendation](https://mlsanity.com/paper/2504.09627/) (2025)
- [Rank-GRPO: Training LLM-based Conversational Recommender Systems with Reinforcement Learning](https://mlsanity.com/paper/2510.20150/) (2025)
- [MR.Rec: Synergizing Memory and Reasoning for Personalized Recommendation Assistant with LLMs](https://mlsanity.com/paper/2510.14629/) (2025)
- [PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization](https://mlsanity.com/paper/2508.07342/) (2025)
- [S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation](https://mlsanity.com/paper/2601.18664/) (2026)
- [Generative Reasoning Recommendation via LLMs](https://mlsanity.com/paper/2510.20815/) (2025)
- [Formula-R1: Incentivizing LLM Reasoning over Complex Tables with Numerical Computation via Formula-Driven Reinforcement Learning](https://mlsanity.com/paper/2505.23667/) (2025)
- [PROMISE: Process Reward Models Unlock Test-Time Scaling Laws in Generative Recommendations](https://mlsanity.com/paper/2601.04674/) (2026)
- [Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training](https://mlsanity.com/paper/2602.00747/) (2026)
- [LLM Agents Enable User-Governed Personalization Beyond Platform Boundaries](https://mlsanity.com/paper/2605.09794/) (2026)
- [SAPO: Step-Aligned Policy Optimization for Reasoning-Based Generative Recommendation](https://mlsanity.com/paper/2605.17648/) (2026)
- [R1-Ranker: Teaching LLM Rankers to Reason](https://mlsanity.com/paper/2506.21638/) (2025)
- [MGFRec: Towards Reinforced Reasoning Recommendation with Multiple Groundings and Feedback](https://mlsanity.com/paper/2510.22888/) (2025)
- [Owen-Shapley Policy Optimization: A Principled RL Algorithm for Generative Search LLMs](https://mlsanity.com/paper/2601.08403/) (2026)
- [Don't Let Bandit Feedback Pull Continual LLM-Recommender Updates Off Target](https://mlsanity.com/paper/2605.18899/) (2026)
- [Self-supervised User Profile Generation for Personalization](https://mlsanity.com/paper/2606.05336/) (2026)
- [RGAlign-Rec: Ranking-Guided Alignment for Latent Query Reasoning in Recommendation Systems](https://mlsanity.com/paper/2602.12968/) (2026)
- [FlexRec: Adapting LLM-based Recommenders for Flexible Needs via Reinforcement Learning](https://mlsanity.com/paper/2603.11901/) (2026)
- [Uncertainty as a Planning Signal: Multi-Turn Decision Making for Goal-Oriented Conversation](https://mlsanity.com/paper/2604.03924/) (2026)
- [Enhancing Local Life Service Recommendation with Agentic Reasoning in Large Language Model](https://mlsanity.com/paper/2604.14051/) (2026)
- [ReCast: Recasting Learning Signals for Reinforcement Learning in Generative Recommendation](https://mlsanity.com/paper/2604.22169/) (2026)
- [RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation](https://mlsanity.com/paper/2605.07129/) (2026)
- [ExpWeaver: LLM Agents Learn from Experience via Latent RAG](https://mlsanity.com/paper/2606.01041/) (2026)
- [Harmonizing Semantic and Collaborative in LLMs: Reasoning-based Embedding Generator for Sequential Recommendation](https://mlsanity.com/paper/2606.16703/) (2026)

## Abstract (arXiv)

We propose Rec-R1, a general reinforcement learning framework that bridges large language models (LLMs) with recommendation systems through closed-loop optimization. Unlike prompting and supervised fine-tuning (SFT), Rec-R1 directly optimizes LLM generation using feedback from a fixed black-box recommendation model, without relying on synthetic SFT data from proprietary models such as GPT-4o. This avoids the substantial cost and effort required for data distillation. To verify the effectiveness of Rec-R1, we evaluate it on two representative tasks: product search and sequential recommendation. Experimental results demonstrate that Rec-R1 not only consistently outperforms prompting- and SFT-based methods, but also achieves significant gains over strong discriminative baselines, even when used with simple retrievers such as BM25. Moreover, Rec-R1 preserves the general-purpose capabilities of the LLM, unlike SFT, which often impairs instruction-following and reasoning. These findings suggest Rec-R1 as a promising foundation for continual task-specific adaptation without catastrophic forgetting.