# SPRec: Self-Play to Debias LLM-based Recommendation

arXiv: https://arxiv.org/abs/2412.09243 · HTML: https://mlsanity.com/paper/2412.09243/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-rewarding alignment and self-training](https://mlsanity.com/synth/self-rewarding-alignment-and-self-training/)

## Builds on (most cited first, 30 of 30)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models](https://mlsanity.com/paper/2401.01335/) (2024)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [Large Language Models are Zero-Shot Rankers for Recommender Systems](https://mlsanity.com/paper/2305.08845/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive](https://mlsanity.com/paper/2402.13228/) (2024)
- [Bias and Debias in Recommender System: A Survey and Future Directions](https://mlsanity.com/paper/2010.03240/) (2020)
- [Self-Play Preference Optimization for Language Model Alignment](https://mlsanity.com/paper/2405.00675/) (2024)
- [Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraints](https://mlsanity.com/paper/2309.16240/) (2023)
- [LLMRec: Large Language Models with Graph Augmentation for Recommendation](https://mlsanity.com/paper/2311.00423/) (2023)
- [AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems](https://mlsanity.com/paper/2310.09233/) (2023)
- [Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation](https://mlsanity.com/paper/2305.07609/) (2023)
- [A Survey on the Fairness of Recommender Systems](https://mlsanity.com/paper/2206.03761/) (2022)
- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [Human Alignment of Large Language Models through Online Preference Optimisation](https://mlsanity.com/paper/2403.08635/) (2024)
- [A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems](https://mlsanity.com/paper/2308.08434/) (2023)
- [Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective](https://mlsanity.com/paper/2404.04626/) (2024)
- [On Softmax Direct Preference Optimization for Recommendation](https://mlsanity.com/paper/2406.09215/) (2024)
- [CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System](https://mlsanity.com/paper/2204.01266/) (2022)
- [Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation](https://mlsanity.com/paper/2307.04571/) (2023)
- [Fairness in Recommendation: Foundations, Methods and Applications](https://mlsanity.com/paper/2205.13619/) (2022)
- [Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning](https://mlsanity.com/paper/2405.15624/) (2024)
- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [On the Theories Behind Hard Negative Sampling for Recommendation](https://mlsanity.com/paper/2302.03472/) (2023)
- [Agentic Feedback Loop Modeling Improves Recommendation and User Simulation](https://mlsanity.com/paper/2410.20027/) (2024)
- [A Study of Implicit Ranking Unfairness in Large Language Models](https://mlsanity.com/paper/2311.07054/) (2023)
- [Unveiling User Satisfaction and Creator Productivity Trade-Offs in Recommendation Platforms](https://mlsanity.com/paper/2410.23683/) (2024)

## Built on by (most cited first, 18 of 18)

- [Fine-grained List-wise Alignment for Generative Medication Recommendation](https://mlsanity.com/paper/2505.20218/) (2025)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Models](https://mlsanity.com/paper/2507.06507/) (2025)
- [Don't Start Over: A Cost-Effective Framework for Migrating Personalized Prompts Between LLMs](https://mlsanity.com/paper/2601.12034/) (2026)
- [Tree of Preferences for Diversified Recommendation](https://mlsanity.com/paper/2601.02386/) (2025)
- [Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders](https://mlsanity.com/paper/2605.16825/) (2026)
- [Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation](https://mlsanity.com/paper/2511.18740/) (2025)
- [Bridging Semantic Understanding and Popularity Bias with LLMs](https://mlsanity.com/paper/2601.09478/) (2026)
- [BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models](https://mlsanity.com/paper/2601.22925/) (2026)
- [Taming the Long Tail: Efficient Item-wise Sharpness-Aware Minimization for LLM-based Recommender Systems](https://mlsanity.com/paper/2603.12752/) (2026)
- [Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation](https://mlsanity.com/paper/2604.27747/) (2026)
- [RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation](https://mlsanity.com/paper/2605.07129/) (2026)
- [DCGL: Dual-Channel Graph Learning with Large Language Models for Knowledge-Aware Recommendation](https://mlsanity.com/paper/2605.07314/) (2026)
- [AgentGR: Semantic-aware Agentic Group Decision-Making Simulator for Group Recommendation](https://mlsanity.com/paper/2605.10367/) (2026)
- [Goal-Conditioned Supervised Learning for LLM Fine-Tuning](https://mlsanity.com/paper/2605.16345/) (2026)
- [LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation](https://mlsanity.com/paper/2607.04270/) (2026)
- [rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation](https://mlsanity.com/paper/2608.18952/) (2026)
- [TATK: Triple-Aware Top-K Learning with Knowledge-Grounded Verification for LLM-based Sequential Recommendation](https://mlsanity.com/paper/2609.14565/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have attracted significant attention in recommendation systems. Current work primarily applies supervised fine-tuning (SFT) to adapt the model for recommendation tasks. However, SFT on positive examples only limits the model's ability to align with user preference. To address this, researchers recently introduced Direct Preference Optimization (DPO), which explicitly aligns LLMs with user preferences using offline preference ranking data. However, we found that DPO inherently biases the model towards a few items, exacerbating the filter bubble issue and ultimately degrading user experience. In this paper, we propose SPRec, a novel self-play framework designed to mitigate over-recommendation and improve fairness without requiring additional data or manual intervention. In each self-play iteration, the model undergoes an SFT step followed by a DPO step, treating offline interaction data as positive samples and the predicted outputs from the previous iteration as negative samples. This effectively re-weights the DPO loss function using the model's logits, adaptively suppressing biased items. Extensive experiments on multiple real-world datasets demonstrate SPRec's effectiveness in enhancing recommendation accuracy and fairness. The implementation is available via https://github.com/RegionCh/SPRec