Causal Direct Preference Optimization for Distributionally Robust Generative Recommendation

arXiv 2603.22335 ↗First posted 21 Mar 2026cs.IR cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Direct Preference Optimization (DPO) guides large language models (LLMs) to generate recommendations aligned with user historical behavior distributions by minimizing preference alignment loss. However, our systematic empirical research and theoretical analysis reveal that DPO tends to amplify spurious correlations caused by environmental confounders during the alignment process, significantly undermining the generalization capability of LLM-based generative recommendation methods in out of distribution (OOD) scenarios. To mitigate this issue, we propose CausalDPO, an extension of DPO that incorporates a causal invariance learning mechanism. This method introduces a backdoor adjustment strategy during the preference alignment phase to eliminate interference from environmental confounders, explicitly models the latent environmental distribution using a soft clustering approach, and enhances robust consistency across diverse environments through invariance constraints. Theoretical analysis demonstrates that CausalDPO can effectively capture users stable preference structures across multiple environments, thereby improving the OOD generalization performance of LLM-based recommendation models. We conduct extensive experiments under four representative distribution shift settings to validate the effectiveness of CausalDPO, achieving an average performance improvement of 17.17% across four evaluation metrics.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 41 of its 57 references that are in the map. First posted on arXiv 21 Mar 2026.

Builds on

41 of 57 references.

Closest ancestors · 1

Same direction, newest first

2024On Softmax Direct Preference Optimization for Recommendation13 Jun

Other references · 40

2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
2017Deep reinforcement learning from human preferencescited by 3,846
2023A Survey on Evaluation of Large Language Modelscited by 1,227
2018Self-Attentive Sequential Recommendationcited by 1,169
2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
2023A Survey on Large Language Models for Recommendationcited by 378
2023How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Compositioncited by 222
2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 171
2025OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignmentcited by 167
2023Representation Learning with Large Language Models for Recommendationcited by 155

28 more

2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
2024Learnable Item Tokenization for Generative Recommendationcited by 106
2024Controllable Preference Optimization: Toward Controllable Multi-Objective Alignmentcited by 105
2023How to Index Item IDs for Recommendation Foundation Modelscited by 102
2023CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendationcited by 99
2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
2020Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender Systemcited by 94
2017Causal Embeddings for Recommendationcited by 79
2019Distributionally Robust Optimization: A Reviewcited by 75
2023A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systemscited by 73
2023Large Language Models for Generative Recommendation: A Survey and Visionary Discussionscited by 72
2023Collaborative Large Language Model for Recommender Systemscited by 63
2023ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Modelscited by 54
2024Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimizationcited by 52
2024Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender Systemcited by 50
2024Offline Regularised Reinforcement Learning for Large Language Models Alignmentcited by 48
2024LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendationcited by 47
2024Graph Out-of-Distribution Generalization via Causal Interventioncited by 30
2024Item-side Fairness of Large Language Model-based Recommendation Systemcited by 21
2024LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendationcited by 18
2024Distributionally Robust Graph-based Recommendation Systemcited by 17
2024Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Modelcited by 13
2024RosePO: Aligning LLM-based Recommenders with Human Valuescited by 9
2024Symmetric Graph Contrastive Learning against Noisy Views for Recommendationcited by 7
2025Distributionally Robust Graph Out-of-Distribution Recommendation via Diffusion Modelcited by 7
2024Few-Shot Causal Representation Learning for Out-of-Distribution Generalization on Heterogeneous Graphscited by 6
2024Active Large Language Model-based Knowledge Distillation for Session-based Recommendationcited by 6
2026Tail-Aware Data Augmentation for Long-Tail Sequential Recommendationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way