# RosePO: Aligning LLM-based Recommenders with Human Values

arXiv: https://arxiv.org/abs/2410.12519 · HTML: https://mlsanity.com/paper/2410.12519/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2024.

## Built on by (most cited first, 9 of 9)

- [RecNet: Self-Evolving Preference Propagation for Agentic Recommender Systems](https://mlsanity.com/paper/2601.21609/) (2026)
- [Large Language Model Empowered Recommendation Meets All-domain Continual Pre-Training](https://mlsanity.com/paper/2504.08949/) (2025)
- [Improving LLM-based Recommendation with Self-Hard Negatives from Intermediate Layers](https://mlsanity.com/paper/2602.17410/) (2026)
- [IceBreaker for Conversational Agents: Breaking the First-Message Barrier with Personalized Starters](https://mlsanity.com/paper/2604.18375/) (2026)
- [On Negative-aware Preference Optimization for Recommendation](https://mlsanity.com/paper/2508.09653/) (2025)
- [MLLMRec-R1: Incentivizing Reasoning Capability in Large Language Models for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2603.06243/) (2026)
- [Causal Direct Preference Optimization for Distributionally Robust Generative Recommendation](https://mlsanity.com/paper/2603.22335/) (2026)
- [Enhancing Local Life Service Recommendation with Agentic Reasoning in Large Language Model](https://mlsanity.com/paper/2604.14051/) (2026)
- [Goal-Conditioned Supervised Learning for LLM Fine-Tuning](https://mlsanity.com/paper/2605.16345/) (2026)

## Abstract (arXiv)

Recently, there has been a growing interest in leveraging Large Language Models (LLMs) for recommendation systems, which usually adapt a pre-trained LLM to the recommendation scenario through supervised fine-tuning (SFT). However, both the pre-training and SFT stages fail to explicitly model the comparative relationships of a user's preferences on different items. To construct a "helpful and harmless" LLM-based recommender, we propose a general framework -- Recommendation with smoothing personalized Preference Optimization (RosePO), which better aligns with customized human values during the post-training stage. Specifically, in addition to the input and chosen response that naturally align with SFT data, we design a rejected sampling strategy tailored for enhancing helpfulness, along with two strategies aimed at mitigating biases to promote harmlessness. To ensure robustness against uncertain labels present in automatically constructed preference data, we introduce a personalized smoothing factor predicted by a preference oracle into the optimization objective. Evaluation on three real-world datasets demonstrates the effectiveness of our method, showcasing not only improved recommendation performance but also mitigation of semantic hallucination and popularity bias.