mlsanityData to 14 Oct 2026

DynamicPO: Dynamic Preference Optimization for Recommendation

arXiv 2605.00327 ↗First posted 1 May 2026cs.IR cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

In large language model (LLM)-based recommendation systems, direct preference optimization (DPO) effectively aligns recommendations with user preferences, requiring multi-negative objective functions to leverage abundant implicit-feedback negatives and sharpen preference boundaries. However, our empirical analyses reveal a counterintuitive phenomenon, preference optimization collapse, where increasing the number of negative samples can lead to performance degradation despite a continuously decreasing training loss. We further theoretically demonstrate that this collapse arises from gradient suppression, caused by the dominance of easily discriminable negatives over boundary-critical negatives that truly define user preference boundaries. As a result, boundary-relevant signals are under-optimized, weakening the model's decision boundary. Motivated by these observations, we propose DynamicPO (Dynamic Preference Optimization), a lightweight and plug-and-play framework comprising two adaptive mechanisms: Dynamic Boundary Negative Selection, which identifies and prioritizes informative negatives near the model's decision boundary, and Dual-Margin Dynamic beta Adjustment, which calibrates optimization strength per sample according to boundary ambiguity. Extensive experiments on three public datasets show that DynamicPO effectively prevents optimization collapse and improves recommendation accuracy on multi-negative preference optimization methods, with negligible computational overhead. Our code and datasets are available at https://github.com/xingyuHuxingyu/DynamicPO.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 21 of its 26 references that are in the map. First posted on arXiv 1 May 2026.

Builds on

21 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024On Softmax Direct Preference Optimization for Recommendation13 Jun

Other references · 20

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2018Self-Attentive Sequential Recommendationcited by 1,184
  3. 2015Session-based Recommendations with Recurrent Neural Networkscited by 695
  4. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 476
  5. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
  6. 2023A Survey on Large Language Models for Recommendationcited by 379
  7. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 337
  8. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 242
  9. 2023Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisitedcited by 124
  10. 2022Towards Universal Sequence Representation Learning for Recommender Systemscited by 121
  11. 2023CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendationcited by 100
  12. 2024β-DPO: Direct Preference Optimization with Dynamic βcited by 90
8 more
  1. 2023LLaRA: Large Language-Recommendation Assistantcited by 56
  2. 2024Customizing Language Models with Instance-wise LoRA for Sequential Recommendationcited by 32
  3. 2024Rethinking Large Language Model Architectures for Sequential Recommendationscited by 27
  4. 2023Large Language Model Can Interpret Latent Space of Sequential Recommendercited by 19
  5. 2024Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulationcited by 14
  6. 2024Improving Sequential Recommendations with LLMscited by 10
  7. 2025Consistency of Responses and Continuations Generated by Large Language Models on Social Mediacited by 7
  8. 2024MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samplescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025On Negative-aware Preference Optimization for Recommendation
  2. 2026Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
  3. 2025What Makes LLMs Effective Sequential Recommenders? A Study on Preference Intensity and Temporal Context
  4. 2026MASS-DPO: Multi-negative Active Sample Selection for Direct Policy Optimization
  5. 2024Direct Preference Optimization for LLM-Enhanced Recommendation Systems
  6. 2026Autoregressive Direct Preference Optimization
  7. 2025FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
  8. 2025MaPPO: Maximum a Posteriori Preference Optimization with Prior Knowledge