Preference Models assume Proportional Hazards of Utilities
Not on the map. Why?
Abstract (arXiv)
Approaches for estimating preferences from human annotated data typically involves inducing a distribution over a ranked list of choices such as the Plackett-Luce model. Indeed, modern AI alignment tools such as Reward Modelling and Direct Preference Optimization are based on the statistical assumptions posed by the Plackett-Luce model. In this paper, I will connect the Plackett-Luce model to another classical and well known statistical model, the Cox Proportional Hazards model and attempt to shed some light on the implications of the connection therein.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 10 references that are in the map. First posted on arXiv 15 Aug 2025.
Builds on
4 of 10 references.
References · 4
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2016Deep Survival Analysiscited by 61
- 2024Preference Learning Algorithms Do Not Learn Preference Rankingscited by 48
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Learning Heterogeneous Preferences
- 2024Strong Preferences Affect the Robustness of Preference Models and Value Alignment
- 2025Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
- 2024Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
- 2025Evaluating and Aligning Human Economic Risk Preferences in LLMs
- 2026MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment
- 2025Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?
- 2022Models of human preference for learning reward functions