# Causal Direct Preference Optimization for Distributionally Robust Generative Recommendation

arXiv: https://arxiv.org/abs/2603.22335 · HTML: https://mlsanity.com/paper/2603.22335/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 41 of its 57 references that are in the map.
- First posted on arXiv 21 Mar 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Preference alignment for recommenders and RAG](https://mlsanity.com/alignment/preference-alignment-for-recommenders-and-rag/)

## Builds on (most cited first, 30 of 41)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition](https://mlsanity.com/paper/2310.05492/) (2023)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment](https://mlsanity.com/paper/2502.18965/) (2025)
- [LLMRec: Large Language Models with Graph Augmentation for Recommendation](https://mlsanity.com/paper/2311.00423/) (2023)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [Learnable Item Tokenization for Generative Recommendation](https://mlsanity.com/paper/2405.07314/) (2024)
- [Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment](https://mlsanity.com/paper/2402.19085/) (2024)
- [How to Index Item IDs for Recommendation Foundation Models](https://mlsanity.com/paper/2305.06569/) (2023)
- [CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation](https://mlsanity.com/paper/2310.19488/) (2023)
- [Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era](https://mlsanity.com/paper/2404.11457/) (2024)
- [Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System](https://mlsanity.com/paper/2010.15363/) (2020)
- [Causal Embeddings for Recommendation](https://mlsanity.com/paper/1706.07639/) (2017)
- [Distributionally Robust Optimization: A Review](https://mlsanity.com/paper/1908.05659/) (2019)
- [A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems](https://mlsanity.com/paper/2308.08434/) (2023)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [Collaborative Large Language Model for Recommender Systems](https://mlsanity.com/paper/2311.01343/) (2023)
- [On Softmax Direct Preference Optimization for Recommendation](https://mlsanity.com/paper/2406.09215/) (2024)
- [ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models](https://mlsanity.com/paper/2305.06566/) (2023)
- [Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization](https://mlsanity.com/paper/2407.07880/) (2024)
- [Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender System](https://mlsanity.com/paper/2404.11343/) (2024)
- [Offline Regularised Reinforcement Learning for Large Language Models Alignment](https://mlsanity.com/paper/2405.19107/) (2024)
- [LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation](https://mlsanity.com/paper/2405.20646/) (2024)

## Abstract (arXiv)

Direct Preference Optimization (DPO) guides large language models (LLMs) to generate recommendations aligned with user historical behavior distributions by minimizing preference alignment loss. However, our systematic empirical research and theoretical analysis reveal that DPO tends to amplify spurious correlations caused by environmental confounders during the alignment process, significantly undermining the generalization capability of LLM-based generative recommendation methods in out of distribution (OOD) scenarios. To mitigate this issue, we propose CausalDPO, an extension of DPO that incorporates a causal invariance learning mechanism. This method introduces a backdoor adjustment strategy during the preference alignment phase to eliminate interference from environmental confounders, explicitly models the latent environmental distribution using a soft clustering approach, and enhances robust consistency across diverse environments through invariance constraints. Theoretical analysis demonstrates that CausalDPO can effectively capture users stable preference structures across multiple environments, thereby improving the OOD generalization performance of LLM-based recommendation models. We conduct extensive experiments under four representative distribution shift settings to validate the effectiveness of CausalDPO, achieving an average performance improvement of 17.17% across four evaluation metrics.