mlsanityData to 7 Oct 2026

SPRec: Self-Play to Debias LLM-based Recommendation

arXiv 2412.09243 ↗First posted 12 Dec 2024cs.IRcited by 18

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

Large language models (LLMs) have attracted significant attention in recommendation systems. Current work primarily applies supervised fine-tuning (SFT) to adapt the model for recommendation tasks. However, SFT on positive examples only limits the model's ability to align with user preference. To address this, researchers recently introduced Direct Preference Optimization (DPO), which explicitly aligns LLMs with user preferences using offline preference ranking data. However, we found that DPO inherently biases the model towards a few items, exacerbating the filter bubble issue and ultimately degrading user experience. In this paper, we propose SPRec, a novel self-play framework designed to mitigate over-recommendation and improve fairness without requiring additional data or manual intervention. In each self-play iteration, the model undergoes an SFT step followed by a DPO step, treating offline interaction data as positive samples and the predicted outputs from the previous iteration as negative samples. This effectively re-weights the DPO loss function using the model's logits, adaptively suppressing biased items. Extensive experiments on multiple real-world datasets demonstrate SPRec's effectiveness in enhancing recommendation accuracy and fairness. The implementation is available via https://github.com/RegionCh/SPRec

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 30 of its 43 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

30 of 43 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Self-Play Preference Optimization for Language Model Alignment1 May
  2. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models2 Jan

Other references · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2018Self-Attentive Sequential Recommendationcited by 1,169
  4. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  5. 2023A Survey on Large Language Models for Recommendationcited by 378
  6. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  7. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  8. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 239
  9. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 239
  10. 2023Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraintscited by 174
  11. 2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 171
  12. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
16 more
  1. 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 113
  2. 2022A Survey on the Fairness of Recommender Systemscited by 102
  3. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  4. 2024Human Alignment of Large Language Models through Online Preference Optimisationcited by 90
  5. 2023A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systemscited by 73
  6. 2024Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspectivecited by 69
  7. 2024On Softmax Direct Preference Optimization for Recommendationcited by 64
  8. 2022CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender Systemcited by 38
  9. 2023Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendationcited by 38
  10. 2022Fairness in Recommendation: Foundations, Methods and Applicationscited by 27
  11. 2024Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learningcited by 27
  12. 2024Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyondcited by 26
  13. 2023On the Theories Behind Hard Negative Sampling for Recommendationcited by 18
  14. 2024Agentic Feedback Loop Modeling Improves Recommendation and User Simulationcited by 16
  15. 2023A Study of Implicit Ranking Unfairness in Large Language Modelscited by 6
  16. 2024Unveiling User Satisfaction and Creator Productivity Trade-Offs in Recommendation Platformscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 13.

18 papers.

Sort:

By area:

  1. 2025Fine-grained List-wise Alignment for Generative Medication Recommendationcited by 5
  2. 2025GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networkscited by 3
  3. 2025GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Modelscited by 3
  4. 2026Don't Start Over: A Cost-Effective Framework for Migrating Personalized Prompts Between LLMscited by 3
  5. 2025Tree of Preferences for Diversified Recommendationcited by 2
  6. 2025Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendationcited by 1
  7. 2026Bridging Semantic Understanding and Popularity Bias with LLMscited by 1
  8. 2026BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Modelscited by 1
  9. 2026Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenderscited by 1
  10. 2026Taming the Long Tail: Efficient Item-wise Sharpness-Aware Minimization for LLM-based Recommender Systemscited by 0
  11. 2026Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendationcited by 0
  12. 2026RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendationcited by 0
6 more
  1. 2026DCGL: Dual-Channel Graph Learning with Large Language Models for Knowledge-Aware Recommendationcited by 0
  2. 2026AgentGR: Semantic-aware Agentic Group Decision-Making Simulator for Group Recommendationcited by 0
  3. 2026Goal-Conditioned Supervised Learning for LLM Fine-Tuningcited by 0
  4. 2026LBR: Towards Mitigating Length Bias in Large Language Models for Recommendationcited by 0
  5. 2026rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendationcited by 0
  6. 2026TATK: Triple-Aware Top-K Learning with Knowledge-Grounded Verification for LLM-based Sequential Recommendationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving LLM-based Recommendation with Self-Hard Negatives from Intermediate Layers
  2. 2024Direct Preference Optimization for LLM-Enhanced Recommendation Systems
  3. 2025On Negative-aware Preference Optimization for Recommendation
  4. 2026DynamicPO: Dynamic Preference Optimization for Recommendation
  5. 2025DeepRec: Towards a Deep Dive Into the Item Space with Large Language Model Based Recommendation
  6. 2024Towards Efficient and Effective Unlearning of Large Language Models for Recommendation
  7. 2025What Makes LLMs Effective Sequential Recommenders? A Study on Preference Intensity and Temporal Context
  8. 2026Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback