mlsanityData to 7 Oct 2026

On Negative-aware Preference Optimization for Recommendation

arXiv 2508.09653 ↗First posted 13 Aug 2025cs.IR cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Recommendation systems leverage user interaction data to suggest relevant items while filtering out irrelevant (negative) ones. The rise of large language models (LLMs) has garnered increasing attention for their potential in recommendation tasks. However, existing methods for optimizing LLM-based recommenders face challenges in effectively utilizing negative samples. Simply integrating large numbers of negative samples can improve ranking accuracy and mitigate popularity bias but often leads to increased computational overhead and memory costs. Additionally, current approaches fail to account for the varying informativeness of negative samples, leading to suboptimal optimization performance. To address these issues, we propose NAPO (Negative-Aware Preference Optimization), an enhanced framework for preference optimization in LLM-based recommendation. NAPO introduces two key innovations: (1) in-batch negative sharing, which expands the pool of negative samples without additional memory overhead, and (2) dynamic reward margin adjustment, which adapts model updates based on the confidence of negative samples. Extensive experiments on three public datasets demonstrate that NAPO outperforms existing methods in both recommendation accuracy and popularity bias reduction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 44 of its 57 references that are in the map. First posted on arXiv 13 Aug 2025.

Builds on

44 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024On Softmax Direct Preference Optimization for Recommendation13 Jun

Other references · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2018Self-Attentive Sequential Recommendationcited by 1,169
  12. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
31 more
  1. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  2. 2015Session-based Recommendations with Recurrent Neural Networkscited by 685
  3. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 473
  4. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
  5. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  6. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
  7. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 321
  8. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  9. 2020Self-supervised Graph Learning for Recommendationcited by 305
  10. 2024Iterative Reasoning Preference Optimizationcited by 244
  11. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  12. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  13. 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 160
  14. 2022M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systemscited by 152
  15. 2024Provably Robust DPO: Aligning Language Models with Noisy Feedbackcited by 133
  16. 2023Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisitedcited by 124
  17. 2023AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systemscited by 116
  18. 2022Learning Vector-Quantized Item Representation for Transferable Sequential Recommenderscited by 112
  19. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96
  20. 2024β-DPO: Direct Preference Optimization with Dynamic βcited by 89
  21. 2023On Generative Agents in Recommendationcited by 88
  22. 2023Towards Open-World Recommendation with Knowledge Augmentation from Large Language Modelscited by 74
  23. 2023Generative Recommendation: Towards Next-generation Recommender Paradigmcited by 69
  24. 2021SimpleX: A Simple and Strong Baseline for Collaborative Filteringcited by 65
  25. 2022On the Effectiveness of Sampled Softmax Loss for Item Recommendationcited by 39
  26. 2020Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systemscited by 38
  27. 2024Customizing Language Models with Instance-wise LoRA for Sequential Recommendationcited by 32
  28. 2023DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential Recommendationcited by 27
  29. 2024Negative Sampling in Recommendation: A Survey and Future Directionscited by 9
  30. 2024RosePO: Aligning LLM-based Recommenders with Human Valuescited by 9
  31. 2024Finetuning Large Language Model for Personalized Rankingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Hard Negative Sampling via Large Language Models for Recommendation
  2. 2025Listwise Preference Alignment Optimization for Tail Item Recommendation
  3. 2026Improving LLM-based Recommendation with Self-Hard Negatives from Intermediate Layers
  4. 2026DynamicPO: Dynamic Preference Optimization for Recommendation
  5. 2025What Makes LLMs Effective Sequential Recommenders? A Study on Preference Intensity and Temporal Context
  6. 2026Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
  7. 2024Preference Diffusion for Recommendation
  8. 2024Direct Preference Optimization for LLM-Enhanced Recommendation Systems